SPB Git
spboucher.ai

spb/wp30_uqo

Public

WP30: runner, chains, fan-in, transport, smoke runs (Qwen, Llama), pilot plan

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Oct 30, 2026) parent fea8e12 Browse files

201 changed files +3,785 −1,269

201 changed files
Unified Split
modified DECISIONS.md +3 −0
@@ -13,3 +13,6 @@ All times America/Toronto unless noted.
13 - **D-007 Qwen2.5 4-bit KV finding (smoke, not a result).** Both the native mlx_lm path (`generate(..., kv_bits=4)`) and the WP30 adapter produce garbage at 4-bit/group-64 for Qwen2.5-1.5B while Llama-3.2-3B survives even 2-bit/group-32 on the smoke memo. The adapter is therefore not the cause; this is the per-token affine quantization of outlier-heavy Qwen keys (the KIVI motivation). Registered as hypothesis H2; a KIVI-inspired per-channel key quantization is scheduled as a *transport-only* variant (B2b) and labelled "inspired". 13 - **D-007 Qwen2.5 4-bit KV finding (smoke, not a result).** Both the native mlx_lm path (`generate(..., kv_bits=4)`) and the WP30 adapter produce garbage at 4-bit/group-64 for Qwen2.5-1.5B while Llama-3.2-3B survives even 2-bit/group-32 on the smoke memo. The adapter is therefore not the cause; this is the per-token affine quantization of outlier-heavy Qwen keys (the KIVI motivation). Registered as hypothesis H2; a KIVI-inspired per-channel key quantization is scheduled as a *transport-only* variant (B2b) and labelled "inspired".
14 - **D-008 Task families.** (i) Synthetic private-fact ledgers (generator `wp30.data.synthetic`, 15 question types per document incl. unanswerable; evaluator-only keys); (ii) public LongBench v1 subsets qasper and 2wikimqa (all 200 items each; hotpotqa reserved as a 16K-extension subset), raw jsonl from THUDM/LongBench `data.zip` because the HF loading script is no longer supported by `datasets` 5.x; (iii) executable tasks (`wp30.data.exec_tasks`, hidden tests in a `python -I` subprocess). Family (ii) is likely in pretraining corpora; family (i) is not. 14 - **D-008 Task families.** (i) Synthetic private-fact ledgers (generator `wp30.data.synthetic`, 15 question types per document incl. unanswerable; evaluator-only keys); (ii) public LongBench v1 subsets qasper and 2wikimqa (all 200 items each; hotpotqa reserved as a 16K-extension subset), raw jsonl from THUDM/LongBench `data.zip` because the HF loading script is no longer supported by `datasets` 5.x; (iii) executable tasks (`wp30.data.exec_tasks`, hidden tests in a `python -I` subprocess). Family (ii) is likely in pretraining corpora; family (i) is not.
15 - **D-009 Remote node.** M3U96b (Mac Studio M3 Ultra, 96 GB) is the second node for genuine two-node transfer: `~/wp30-worker/.venv` with the same mlx/mlx-lm pins; both 3B checkpoints downloaded there. iperf3 available on both ends for link measurement. 15 - **D-009 Remote node.** M3U96b (Mac Studio M3 Ultra, 96 GB) is the second node for genuine two-node transfer: `~/wp30-worker/.venv` with the same mlx/mlx-lm pins; both 3B checkpoints downloaded there. iperf3 available on both ends for link measurement.
16+- **D-010 Exec family difficulty (dev-only decision, 2026-10-30).** With the 7-rule specification the receiver (Qwen2.5-3B) failed even with the full ticket (A1 pass rate 0 on the smoke item: it duplicated the loop instead of composing the discount rule), so the family could not discriminate between communication methods. The specification was reduced to 6 rules with two missing in the buggy code (region exclusion, tax). Hidden tests unchanged in form (5 subsets, exact float match). Datasets regenerated before any test-split evaluation.
17+- **D-011 Network path for E2/E6 (2026-10-30).** The laptop cannot open TCP connections to the nodes' LAN addresses (192.168.2.x; its SSH goes through Tailscale, whose ACL blocks non-SSH ports). Genuine two-node measurements therefore run **between two cluster nodes on the wired LAN**: sender/client on M3U96a (192.168.2.80), receiver worker on M3U96b (192.168.2.85). Measured link: iperf3 4 streams ≈ 940 Mbit/s (1 GbE, not 10 GbE as older notes suggested), RTT ≈ 1.2 ms. Both nodes concurrently host a CPU-light workload from another study (WP29 `claude --bare` executors); load average ≈ 1.8 on 32 cores at setup time; recorded as a limitation.
18+- **D-012 Learned mapper scope (Track C).** Level 2 (same family, different size): Qwen2.5-1.5B → Qwen2.5-3B share the tokenizer (token-aligned), both have 2 KV heads × 128 dims but 28 vs 36 layers → per-receiver-layer linear maps from an aligned sender layer, trained on train-split caches only (reconstruction loss; optional distillation). Level 3 (cross-family Llama-3.2-3B → Qwen2.5-3B) requires tokenizer alignment by character spans; attempted only if level 2 completes; labelled "adapted (SwaT-inspired), not a SwaT reproduction" (SwaT has no public checkpoint or licence).
modified STATUS.md +5 −4
@@ -1,7 +1,8 @@
1 # STATUS.md — WP30 1 # STATUS.md — WP30
2 2
3−**Phase:** 0 (environment + scope) — in progress 3+**Phase:** 3→4 complete (harness + live inference + cache correctness suite 12/12), 5 in progress (text methods implemented, smoke-tested), 6 pending
4−**Completed gates:** none yet 4+**Completed gates:** Phase 0 (environment, scope), Phase 1 (literature audit: 34 sources, 3 audit files), Phase 2 (datasets: 800 items, keys isolated, 0 cross-split duplicates), Phase 3 (E0 smoke: 114 trials, real generation), Phase 4 (tests/test_cache_correctness.py 12 passed)
5−**Current task:** environment inspection, literature audit launch, model download 5+**Current task:** pilot config, Llama receiver smoke, chain/fan-in smoke, remote worker (M3U96b) for E2/E6
6 **Blockers:** none 6 **Blockers:** none
7−**Next command:** `python -m wp30 doctor --output evidence/environment.json` (after package scaffold) 7+**Resume:** `source .venv/bin/activate && python -m wp30 run --config configs/pilot.yaml --resume`
8+**Run IDs so far:** E0_smoke (runs/raw/E0_smoke.jsonl)
added configs/chains_smoke.yaml +11 −0
@@ -0,0 +1,11 @@
1+experiment_id: E4_smoke
2+splits: [dev]
3+lengths: [2000]
4+limit_docs: 1
5+hops: [1, 2]
6+budget_tokens: 512
7+max_fetch_bytes: 1500
8+max_lookups: 4
9+chains:
10+ - {representation: A4, mode: closed, agents: [qwen3b]}
11+ - {representation: A5, mode: repairable, agents: [qwen3b, llama3b]}
added configs/fanin_smoke.yaml +9 −0
@@ -0,0 +1,9 @@
1+experiment_id: E5_smoke
2+splits: [dev]
3+lengths: [2000]
4+limit_docs: 2
5+fan_in: [2]
6+budget_tokens: 512
7+sender: qwen3b
8+aggregator: qwen3b
9+methods: [full_context, concat_A4, merge_A5, shared_memory_A6, cache_concat_negative_control]
added configs/pilot.yaml +43 −0
@@ -0,0 +1,43 @@
1+experiment_id: E1_pilot
2+splits: [dev]
3+families: [synthetic, exec, public_qasper, public_2wikimqa]
4+lengths: [2000, 8000]
5+limit_docs_per_family: 6
6+max_questions_per_doc: 8
7+model_pairs:
8+ - {sender: qwen3b, receiver: qwen3b}
9+ - {sender: llama3b, receiver: llama3b}
10+generation: {max_tokens: 48, per_family: {exec: 320}}
11+seeds: [0]
12+conditions:
13+ - {method: A0}
14+ - {method: A1}
15+ - {method: A2, budget_tokens: 128}
16+ - {method: A2, budget_tokens: 512}
17+ - {method: A3, budget_tokens: 128, access: blind}
18+ - {method: A3, budget_tokens: 512, access: blind}
19+ - {method: A3, budget_tokens: 512, access: aware}
20+ - {method: A4, budget_tokens: 128, access: blind}
21+ - {method: A4, budget_tokens: 512, access: blind}
22+ - {method: A4, budget_tokens: 512, access: aware}
23+ - {method: A5, budget_tokens: 128, access: blind}
24+ - {method: A5, budget_tokens: 512, access: blind}
25+ - {method: A5, budget_tokens: 512, access: aware}
26+ - {method: A6, budget_tokens: 512, access: blind, max_fetch_bytes: 2000, max_lookups: 6}
27+ - {method: A7, codec: zstd}
28+ - {method: A8, budget_tokens: 512, access: blind}
29+ - {method: B0}
30+ - {method: B1}
31+ - {method: B2, bits: 8}
32+ - {method: B2, bits: 4}
33+ - {method: B2, bits: 2, group_size: 32}
34+ - {method: B3, keep_frac: 0.5}
35+ - {method: B3, keep_frac: 0.25}
36+ - {method: B3, keep_frac: 0.125}
37+ - {method: B4, keep_frac: 0.25}
38+ - {method: B5, keep_frac: 0.5, access: aware}
39+ - {method: B5, keep_frac: 0.25, access: aware}
40+ - {method: B5, keep_frac: 0.125, access: aware}
41+ - {method: B5, keep_frac: 0.25, access: blind}
42+ - {method: B6, keep_frac: 0.25, access: aware}
43+ - {method: B9, keep_frac: 0.25, bits: 8, access: aware, policy: attention}
modified configs/smoke.yaml +1 −1
@@ -6,7 +6,7 @@ limit_docs_per_family: 1
6 max_questions_per_doc: 4 6 max_questions_per_doc: 4
7 model_pairs: 7 model_pairs:
8 - {sender: qwen3b, receiver: qwen3b} 8 - {sender: qwen3b, receiver: qwen3b}
9−generation: {max_tokens: 48} 9+generation: {max_tokens: 48, per_family: {exec: 320}}
10 seeds: [0] 10 seeds: [0]
11 conditions: 11 conditions:
12 - {method: A0} 12 - {method: A0}
modified data/generated/docs/exec/exec_00001_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00001_2000", 2 "document_id": "exec_00001_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 47.53, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 4.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 25, \"unit_price\": 71.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 9.99, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 50.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 26.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 47.53, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 4.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 25, \"unit_price\": 71.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 9.99, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 50.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 26.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8073, 6+ "char_len": 7894,
7− "document_hash": "ddfe7fcc3baca3e97144b66188f285a0e8a83018d1ca41b7f37cb80dede62dcc", 7+ "document_hash": "72a702cc7076fb313f97772498675d83532bf6ce59f1d3cbf3bb86fa387dc0e6",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00002_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00002_2000", 2 "document_id": "exec_00002_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 5.2, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 14.07, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 51.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 37.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 67.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 5.2, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 14.07, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 51.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 37.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 67.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8001, 6+ "char_len": 7819,
7− "document_hash": "7918e3cdcab2a8bfae77558b7b523f4cea70cdc52435393d98d0c7b36de29e60", 7+ "document_hash": "7ec85eb17fda1c57c2a77a31dc79bf35b3c62974f6f1905e409880cbe869b6f5",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00003_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00003_2000", 2 "document_id": "exec_00003_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 35.12, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 47.96, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 27.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 8.05, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 37.32, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 35.12, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 47.96, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 27.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 8.05, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 37.32, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7972, 6+ "char_len": 7793,
7− "document_hash": "8ab845867bddfdcb5cf63b1319a6f29794cc9dddea45cd92bca3bc94fe023da7", 7+ "document_hash": "f7d729b32469d142625681408a1dd1aa3d1d99a7c07adb645c9f40d7550e8695",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00004_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00004_2000", 2 "document_id": "exec_00004_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 66.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 75.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 70.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 56.79, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 16, \"unit_price\": 23.1, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 64.58, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 66.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 75.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 70.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 56.79, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 16, \"unit_price\": 23.1, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 64.58, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 8037, 6+ "char_len": 7855,
7− "document_hash": "08c5cdfaa7a0742a094ee4726d97a32a89395c354270406bbcf78eff4fdbe8b2", 7+ "document_hash": "aeca575de77be2cf25f1c9f8e053485ad918ef4b3468680a4ef6bfa3b8cb781b",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00005_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00005_2000", 2 "document_id": "exec_00005_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 9.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 9.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 26.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 12.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 4, \"unit_price\": 49.06, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 38.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 9.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 9.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 26.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 12.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 4, \"unit_price\": 49.06, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 38.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 8031, 6+ "char_len": 7852,
7− "document_hash": "f885d791d02ed1adb010c9c27804b5c38facf8fd2a09b9b1b6f9cacb7b0acc81", 7+ "document_hash": "c5e26772985f1897f27e3729a56bdf95e08ba4718b928a1480572409c422c83f",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00006_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00006_2000", 2 "document_id": "exec_00006_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 51.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 16.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 75.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 55.85, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 45.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 32.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 51.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 16.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 75.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 55.85, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 45.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 32.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8049, 6+ "char_len": 7867,
7− "document_hash": "2a1bb6d86c77861b166e281bb1deac9a658ed4383a7ade93e9698e33861de0e5", 7+ "document_hash": "283461ef1dfaf2225620026d081a07116ec5fa2173ffa118704c0cf1d53a2eac",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00007_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00007_2000", 2 "document_id": "exec_00007_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 15.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 21.54, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 27.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 48.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 59.99, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 79.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 9}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 15.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 21.54, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 27.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 48.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 59.99, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 79.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 9}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 7826, 6+ "char_len": 7644,
7− "document_hash": "fb0150ce17a4b0ade1601cf6082b82e83f67db83a57c177064777d028b33d97c", 7+ "document_hash": "961b89accd7b0332a165c291bb3c527fd00bea183807cff965326b5f758f1b7f",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00008_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00008_2000", 2 "document_id": "exec_00008_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 27.18, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 55.08, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 28.44, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 59.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 79.85, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 26.43, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 27.18, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 55.08, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 28.44, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 59.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 79.85, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 26.43, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 8052, 6+ "char_len": 7873,
7− "document_hash": "04f896fa681f3ae7cbfc7efa30fd96c43be9d9b703dab76457ee8043483b3612", 7+ "document_hash": "4ad9a1fbe7db8f523841d367742505a5c92581010a15fdabb2bb6b6d77afbaa4",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00009_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00009_2000", 2 "document_id": "exec_00009_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 10.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 5.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 46.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 44.75, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 42.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 10.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 5.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 46.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 44.75, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 42.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8097, 6+ "char_len": 7916,
7− "document_hash": "db44cbab6b33239dddaa205976960167be792c1b1025bd39a4c4d6cf0f6ab7d7", 7+ "document_hash": "7c7609e3e3780713446d7ec4cbe48dba3ad69a1bf6d96393eb7e34fe13581d81",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00010_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00010_2000", 2 "document_id": "exec_00010_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 25.77, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 67.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 78.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 47.19, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 61.6, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 64.37, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 25.77, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 67.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 78.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 47.19, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 61.6, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 64.37, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 8004, 6+ "char_len": 7825,
7− "document_hash": "232fe52471a905a3ae14b591781c17dfa57a2d115258bfc1f8a1fce231b705ad", 7+ "document_hash": "2951e556f1045edd4bb9926de1dac81793260ddcd823e8c6abe38566b4cbcc07",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00011_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00011_2000", 2 "document_id": "exec_00011_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 66.05, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 28.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 44.45, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 7, \"unit_price\": 66.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 6.84, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 64.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 66.05, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 28.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 44.45, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 7, \"unit_price\": 66.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 6.84, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 64.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8135, 6+ "char_len": 7953,
7− "document_hash": "f046c8292307adf97cae067d41f5a7bcce9d2a48bc8dd143e0f11e89a1c3a4c2", 7+ "document_hash": "6b5b870bbcdf744c667890cf6185717b10f87e45ad7a6ccada501987d53f5e6d",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00012_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00012_2000", 2 "document_id": "exec_00012_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 45.84, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 28.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 22.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 52.18, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 46.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 45.84, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 28.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 22.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 52.18, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 46.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 7906, 6+ "char_len": 7724,
7− "document_hash": "2befcba5bfd5b509a9a6ec5d9334e2e384a05b6733ac02d80222302d3ca6e6ef", 7+ "document_hash": "8991dd153dfd83b4a1aea0eb4340aabe4370ac72ef16427f5f67894ac175edde",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00013_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00013_2000", 2 "document_id": "exec_00013_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 19.02, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 31.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 6.76, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 50.8, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 70.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 29.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 19.02, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 31.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 6.76, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 50.8, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 70.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 29.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8051, 6+ "char_len": 7870,
7− "document_hash": "d006207e385cff8703be29071faf5845de84d05d8a677e40dda4983739115d29", 7+ "document_hash": "ee93ee22294a4463cb6cf9ca3d5ab7ea7fd2a991b2b55448f089c2290e51a174",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00014_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00014_2000", 2 "document_id": "exec_00014_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 20.49, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 61.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 28.29, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 53.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 5.5, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 60.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 20.49, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 61.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 28.29, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 53.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 5.5, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 60.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 8089, 6+ "char_len": 7910,
7− "document_hash": "e6ac9234df3fc7dcec86770e041bf75b576d1c65cf6fe32717891429f5461066", 7+ "document_hash": "b25f9d4fb9c997b0cb6e19983b48490097f6592b8d17e66f305f5a1b3e8a8443",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00015_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00015_2000", 2 "document_id": "exec_00015_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 44.28, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 77.11, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 37.27, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 24.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 49.63, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 44.28, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 77.11, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 37.27, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 24.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 49.63, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8041, 6+ "char_len": 7859,
7− "document_hash": "7bd733e903db470812271aa82f1cd9c7111ace0dcd257ace35b07e660d045be7", 7+ "document_hash": "7334bc5b260e11b2b0cdf9db05839cac2f397ae0e44db4e8bba7559dc94036bb",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00016_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00016_2000", 2 "document_id": "exec_00016_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 16.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 13.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 31.16, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 76.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 43.73, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 41.2, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 16.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 13.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 31.16, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 76.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 43.73, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 41.2, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8045, 6+ "char_len": 7864,
7− "document_hash": "b232fee620440e55facf033b892741387c97661b269dc2371313fb0f5934ff69", 7+ "document_hash": "6a5268852caa8dc3015fd42e678e912f463ba061ac417b09de9ac6ed45e85d7f",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00017_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00017_2000", 2 "document_id": "exec_00017_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 71.87, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 57.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 63.43, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 39, \"unit_price\": 73.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 41.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 17, \"unit_price\": 36.3, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 71.87, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 57.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 63.43, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 39, \"unit_price\": 73.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 41.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 17, \"unit_price\": 36.3, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8053, 6+ "char_len": 7872,
7− "document_hash": "bce532bd3702b32e6eb3620ef3ea7d860a695f22bb90b8e995836a7937c77f52", 7+ "document_hash": "f2d06422bd98b400b7401ce4c5ae2939aeb43f51482e84cba0c0e3c0845c71a5",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00018_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00018_2000", 2 "document_id": "exec_00018_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 62.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 68.54, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 63.9, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 71.94, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 9.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 26.85, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 62.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 68.54, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 63.9, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 71.94, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 9.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 26.85, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 8169, 6+ "char_len": 7990,
7− "document_hash": "6f0951d7af338593059efbe431f0b7ce5bf9ae652e3dfb9eedafa7e192b80c01", 7+ "document_hash": "f219110fda4e5d20fabbe592248da3e78f61f411dbc16b9412d9b8d692eda870",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00019_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00019_2000", 2 "document_id": "exec_00019_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 55.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 5.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 35.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 18.49, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 38.49, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 40, \"unit_price\": 46.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 55.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 5.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 35.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 18.49, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 38.49, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 40, \"unit_price\": 46.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8031, 6+ "char_len": 7849,
7− "document_hash": "733853d52a559dfc67d0cdab02de9e21c2aa09e0b25702d2cadfb38c20ab2159", 7+ "document_hash": "232819b55ebc60215845a14aaf01c04a94dc7902933c03a47546ec41ee16502f",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00020_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00020_2000", 2 "document_id": "exec_00020_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 77.87, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 51.72, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 67.82, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 22.1, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 49.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 77.87, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 51.72, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 67.82, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 22.1, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 49.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 8080, 6+ "char_len": 7901,
7− "document_hash": "1a7bfffc3347254640c8b76b9b5f0ff368a758543995fa06006e28ea43d8eb71", 7+ "document_hash": "862136f6ccb7f77097086ceddd84945e55b7df17401a63b0ce306740969966e5",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00101_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00101_2000", 2 "document_id": "exec_00101_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 53.66, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 17.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 66.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 33.18, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 47.37, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 22.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 53.66, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 17.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 66.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 33.18, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 47.37, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 22.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8125, 6+ "char_len": 7944,
7− "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", 7+ "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00102_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00102_2000", 2 "document_id": "exec_00102_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 68.35, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 65.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 17, \"unit_price\": 8.56, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 36, \"unit_price\": 48.5, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 22, \"unit_price\": 76.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 51.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 68.35, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 65.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 17, \"unit_price\": 8.56, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 36, \"unit_price\": 48.5, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 22, \"unit_price\": 76.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 51.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8089, 6+ "char_len": 7910,
7− "document_hash": "5535ac3109616f7d46dff5417535ca232754aee2c0759e072d552e465f004d46", 7+ "document_hash": "61b6c7bb86261a26bf000996edd5f9326f7f77ca1c37085fd999cddf1bff2396",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00103_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00103_2000", 2 "document_id": "exec_00103_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 24.8, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 13.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 26.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 10.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 34.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 17.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 24.8, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 13.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 26.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 10.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 34.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 17.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8038, 6+ "char_len": 7856,
7− "document_hash": "a7875d1e95742c6246c4ec3862b9797a4269a2b607e47869f5014d7dc29139e0", 7+ "document_hash": "596ef1cd6d37fdac729520a738ed129e4ffaf619d372ad869d8188a0a5cac1ef",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00104_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00104_2000", 2 "document_id": "exec_00104_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 26.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 59.29, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 50.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 74.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 40.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 26.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 59.29, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 50.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 74.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 40.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8014, 6+ "char_len": 7835,
7− "document_hash": "bce3091895fc315bf981ac1272a7da1f00b0f64819f1ecfa2a6dd8a3d23f2207", 7+ "document_hash": "6b4aadaf1b71672d6d8aed5a412c7aa490e3e0eb1c92e5df6db1cc9554f83483",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00105_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00105_2000", 2 "document_id": "exec_00105_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 44.66, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 12.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 71.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 69.08, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 71.78, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 17.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 44.66, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 12.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 71.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 69.08, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 71.78, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 17.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8046, 6+ "char_len": 7867,
7− "document_hash": "fe4b8857bf1774be1123bec85a5743da5c3551d53125ddf128efde401f3990c2", 7+ "document_hash": "a59face90fe592d3d8a9b375aafefb775631dd798a60a424f981cceae1ec2996",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00106_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00106_2000", 2 "document_id": "exec_00106_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 66.67, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 69.05, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 67.57, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 49.2, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 40.78, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 31, \"unit_price\": 40.56, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 66.67, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 69.05, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 67.57, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 49.2, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 40.78, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 31, \"unit_price\": 40.56, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 7943, 6+ "char_len": 7761,
7− "document_hash": "c3652bfa1632f0cc62d6512131ef519100143881225c8b4ee572c1876e7c6c7f", 7+ "document_hash": "a78f55af374fd04eb589b193a9eca2f84cf04bcdb3fa60aaeb29fbe358c79464",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00107_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00107_2000", 2 "document_id": "exec_00107_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.74, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 4.67, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 67.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 10.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 61.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 16.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.74, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 4.67, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 67.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 10.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 61.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 16.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 7930, 6+ "char_len": 7751,
7− "document_hash": "983dbcacdfae2acf0316690a7fe50cfa2523ae41beffd9de42356f2ebd3d1e79", 7+ "document_hash": "83ea6e4f6c5d3a6eed7a921a930819d760ffb3dac7c22761dc9b4c28084904cc",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00108_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00108_2000", 2 "document_id": "exec_00108_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 59.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 47.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 24.32, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 53.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 33.13, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 58.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 59.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 47.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 24.32, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 53.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 33.13, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 58.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 8038, 6+ "char_len": 7859,
7− "document_hash": "7ebb05d3714e237d074dfc9a69c5b83e99932f9041b5d4fe15c8fc7224ba7078", 7+ "document_hash": "b9ccaaf72598ea6e12a10942f2697dd01ca891115b652043a2f6ce4cb68570cc",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00109_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00109_2000", 2 "document_id": "exec_00109_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 16.34, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 77.14, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 21.7, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 65.34, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 36.24, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.17, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 16.34, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 77.14, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 21.7, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 65.34, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 36.24, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.17, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8073, 6+ "char_len": 7894,
7− "document_hash": "680dab47e67683cb351941eea647197cb2b2ea472ff116582019ace0c543d461", 7+ "document_hash": "d53ada6a135b4827dcacb2ebec142b5000ad21961fb0cd199fc7c609424d4d5a",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00110_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00110_2000", 2 "document_id": "exec_00110_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 48.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.65, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 52.8, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 30.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 18.59, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 26.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 48.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.65, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 52.8, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 30.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 18.59, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 26.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 8019, 6+ "char_len": 7840,
7− "document_hash": "f1699da28903736ad58ec0640a277bb2eac030c80c49c22c027c273374df86fa", 7+ "document_hash": "789f1f1a750d514137a988d21a0807cc30c8548da830f2eeef66854fc179fcf5",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00111_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00111_2000", 2 "document_id": "exec_00111_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 74.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 62.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 46.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 49.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 62.57, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 38.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 74.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 62.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 46.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 49.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 62.57, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 38.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 7942, 6+ "char_len": 7763,
7− "document_hash": "e65e9534b60b15cdaadf465396c6385868f5a0ed4be5ba04c458937c2eff43ae", 7+ "document_hash": "9f3d9d2caa1830b0bf0b983179f4de3251ec9bb8648c710bf43f0fb6c531ef85",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00112_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00112_2000", 2 "document_id": "exec_00112_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"gamma\", \"units\": 4, \"unit_price\": 38.52, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 13, \"unit_price\": 23.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 18.69, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 55.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 19.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 61.42, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"gamma\", \"units\": 4, \"unit_price\": 38.52, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 13, \"unit_price\": 23.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 18.69, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 55.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 19.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 61.42, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6− "char_len": 7996, 6+ "char_len": 7814,
7− "document_hash": "4a8db801f5e7f6b2108618514f90c37fbdbb61e902c4441c0d68548a5792da05", 7+ "document_hash": "182cf1cbb5939b7e1ca9170096aa77a9cc3c2c4f8efb3195e8b4ad19b77a3f46",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00113_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00113_2000", 2 "document_id": "exec_00113_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 39.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 5.13, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 20.94, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 3.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 42.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 3, \"unit_price\": 57.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 39.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 5.13, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 20.94, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 3.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 42.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 3, \"unit_price\": 57.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 8055, 6+ "char_len": 7874,
7− "document_hash": "7535b4d324a3e28b457e40fe41d18d52e75a9ed0271b411285e522c6c8c8200e", 7+ "document_hash": "c89720bd112bca637c8fa3b4a98f7cc41a384be468eee47f5c44607ee6fc46f1",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00114_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00114_2000", 2 "document_id": "exec_00114_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 39.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 65.95, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 38, \"unit_price\": 38.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.5, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.46, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 10.31, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 6}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 39.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 65.95, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 38, \"unit_price\": 38.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.5, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.46, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 10.31, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 6}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8042, 6+ "char_len": 7863,
7− "document_hash": "e1faab8175075b8cf98d2e5e16181db3f5a8f103ed89d35efee7ba4684968699", 7+ "document_hash": "86a377fe8f0559479ca21128972c50d14bd9e19d38d2bf49f763d402259c0aad",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00115_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00115_2000", 2 "document_id": "exec_00115_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 62.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 35, \"unit_price\": 64.7, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 29.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 17.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 53.93, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 62.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 35, \"unit_price\": 64.7, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 29.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 17.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 53.93, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6− "char_len": 8127, 6+ "char_len": 7946,
7− "document_hash": "4d6a9aa137c992493ef42f6758dbb35b873a4a392b00eae7152d6e45b68a3c1b", 7+ "document_hash": "33ee1d20ea5713048e1335d3786ba0b10b3a5998a16200f09823d0c4cf0dcbe1",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00116_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00116_2000", 2 "document_id": "exec_00116_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 40.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 3, \"unit_price\": 67.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 21.59, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 21.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 30.51, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 66.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 40.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 3, \"unit_price\": 67.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 21.59, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 21.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 30.51, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 66.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 7941, 6+ "char_len": 7762,
7− "document_hash": "2b6100ff4b04865ebf622b9749757f3b32215faa2e32f12d63be71b0c18fc7be", 7+ "document_hash": "379dcf9e9ada65463c8d73e2cbb54ff33c8c46ea3154539a1d6df25bab2a0a0e",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00117_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00117_2000", 2 "document_id": "exec_00117_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 34.17, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 39.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 37.26, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 13.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 4.12, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 1, \"unit_price\": 4.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 34.17, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 39.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 37.26, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 13.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 4.12, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 1, \"unit_price\": 4.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 8064, 6+ "char_len": 7883,
7− "document_hash": "6e53d886a35d8e51c98908f1e40e9314353c62a19e3197603509fd2a59c6bbf3", 7+ "document_hash": "34a1cc9614d0c2d2f9766b2968ae3fa3eb2b6cc1a372750650ae7f7359325896",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00118_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00118_2000", 2 "document_id": "exec_00118_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 68.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 35.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 65.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 2, \"unit_price\": 53.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 48.57, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 68.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 35.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 65.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 2, \"unit_price\": 53.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 48.57, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 8050, 6+ "char_len": 7868,
7− "document_hash": "5353584056feb451c43bd2199aa87e2f7064bbfb9485b6eb1f6a1387dddac0f6", 7+ "document_hash": "58d2108f993d9baafb67222dc4bd97895639cb6c2410e675e3ed2c4233c8fee4",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00119_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00119_2000", 2 "document_id": "exec_00119_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.28, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 28.27, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 56.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 58.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 21, \"unit_price\": 7.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 10.91, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.28, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 28.27, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 56.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 58.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 21, \"unit_price\": 7.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 10.91, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 7901, 6+ "char_len": 7722,
7− "document_hash": "f62c9b5ecd91caa0cf95af49bf6d40dd2bc3885255008434f482e56b7a2a2fae", 7+ "document_hash": "98f121e7a33ff30acfcaf7a0346bbc04cc9f508e092e316490793765a8e0ab09",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00120_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00120_2000", 2 "document_id": "exec_00120_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 26, \"unit_price\": 76.39, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.78, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 55.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 48.6, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 31.92, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 33.32, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 26, \"unit_price\": 76.39, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.78, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 55.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 48.6, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 31.92, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 33.32, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7992, 6+ "char_len": 7811,
7− "document_hash": "5b1b1089a793b0953579b8ce004efe99282ffd8a6eb17b16c3ce50273c121aad", 7+ "document_hash": "c5be33e5009123ba2c251884304b839ae6fc5163f0745c302cffbabf618b40cb",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00121_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00121_2000", 2 "document_id": "exec_00121_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 48.97, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 58.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 78.34, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 65.31, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 52.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 37, \"unit_price\": 9.99, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 48.97, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 58.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 78.34, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 65.31, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 52.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 37, \"unit_price\": 9.99, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6− "char_len": 8033, 6+ "char_len": 7854,
7− "document_hash": "9064e893e331010c27db6e1229c2af7da83ed7fb3680e079fc5c1a9ee7b2d73a", 7+ "document_hash": "1645df3901b5bd61ea8c6790e15a695b1144eef69bc075121ffd0c83e6fc13af",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00122_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00122_2000", 2 "document_id": "exec_00122_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 43.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 74.34, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 31.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 12.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 24.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 38, \"unit_price\": 69.36, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 43.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 74.34, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 31.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 12.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 24.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 38, \"unit_price\": 69.36, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8139, 6+ "char_len": 7960,
7− "document_hash": "8641f3a09b033f2a216948ea197682c280cda7a9559b7102473da5f152a73eb3", 7+ "document_hash": "bd26e436a86876d04961cc142588671db4ead753f838933bde6907da5354f70e",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00123_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00123_2000", 2 "document_id": "exec_00123_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 44.11, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 43.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 27.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 15, \"unit_price\": 68.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 49.94, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 57.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 44.11, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 43.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 27.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 15, \"unit_price\": 68.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 49.94, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 57.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 8069, 6+ "char_len": 7890,
7− "document_hash": "3a8a85801b1200bb0a18063e20eb67097e3e9c34d009894a5a61963f7145608b", 7+ "document_hash": "6c868c5916de1b984d70dc327ee458814726f8ea6a1fad99f8c2fb35e7e77b7f",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00124_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00124_2000", 2 "document_id": "exec_00124_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 15.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 16.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 32.63, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 22.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 53.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 66.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 15.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 16.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 32.63, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 22.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 53.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 66.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 7919, 6+ "char_len": 7740,
7− "document_hash": "adfa94aa242540ae900f58c2db73dace6e0bb6ec243a71137685b2d527e54fdb", 7+ "document_hash": "a285cd3f7f5bbfcaa508697bf261341dc847483a97e354b95faf09bc23bb626f",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00125_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00125_2000", 2 "document_id": "exec_00125_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 74.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 79.31, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 24.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 61.06, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 48.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 73.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 74.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 79.31, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 24.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 61.06, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 48.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 73.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 7915, 6+ "char_len": 7733,
7− "document_hash": "144a4af055228fc4045ad15b1ca9ba8229c8e5563cc4c4cf9ad94203a0072dd9", 7+ "document_hash": "386fcaff7daca426267d33af6f30657f21e4863b66d5a71d38d0b78a334eb069",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00126_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00126_2000", 2 "document_id": "exec_00126_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 14.21, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 40.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 66.39, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 5, \"unit_price\": 58.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 35.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 51.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 14.21, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 40.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 66.39, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 5, \"unit_price\": 58.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 35.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 51.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 7944, 6+ "char_len": 7763,
7− "document_hash": "a6bed1ca1c6bee001ded092712010d4dc8d3ec10d0d1f85432e8844be28b9359", 7+ "document_hash": "0729452f4b76415c15bdd0075e98c4db16464515a675ea3d8e3611adb3c8a3e9",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00127_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00127_2000", 2 "document_id": "exec_00127_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 4.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 57.56, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 48.4, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 7.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 49.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.89, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 4.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 57.56, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 48.4, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 7.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 49.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.89, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 7953, 6+ "char_len": 7774,
7− "document_hash": "01e9b8b715cbfdd0d66c26d571044c18f9a6d9d2c5146b834291716879cf02d2", 7+ "document_hash": "bb6e6467042a1b7809f3ea38b83ea82e5554ca36b70042023f682efd5bae752a",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00128_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00128_2000", 2 "document_id": "exec_00128_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 33.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 74.94, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 71.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 69.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 38.14, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 33.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 74.94, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 71.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 69.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 38.14, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 7945, 6+ "char_len": 7763,
7− "document_hash": "e702731f232f4b91d0513add4b301fd1ef0382eee35df22a98cef7eb727fafca", 7+ "document_hash": "ea7fe68afa60ebc43cd0a501c243c72cd3b993bfaef6e389f12308e1967e3471",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00129_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00129_2000", 2 "document_id": "exec_00129_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 31.69, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 44.06, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 28.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 18.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 32.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 10.85, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 31.69, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 44.06, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 28.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 18.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 32.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 10.85, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6− "char_len": 7973, 6+ "char_len": 7794,
7− "document_hash": "e508132470ec83c8745c1899d6c954b865130cad33382718fa48d35d6bea0ec8", 7+ "document_hash": "46f3b799f18b08ee443481845e8695eff10ac2f2b23c2b177eafce3b42e2ea40",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00130_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00130_2000", 2 "document_id": "exec_00130_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 4.14, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 3.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 18.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 23.95, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 17.39, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 71.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 4.14, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 3.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 18.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 23.95, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 17.39, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 71.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8072, 6+ "char_len": 7893,
7− "document_hash": "c65f14fbcb1d4d0049a6e641be2826817637866851032f95c4cc1943f87ff809", 7+ "document_hash": "c415053d25119b89bd769d02be02c390efebcd68b992b946ddf043ad29041f86",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00201_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00201_2000", 2 "document_id": "exec_00201_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 5.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 15.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 54.9, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 7.38, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 33, \"unit_price\": 19.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 43.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 5.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 15.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 54.9, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 7.38, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 33, \"unit_price\": 19.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 43.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 8073, 6+ "char_len": 7892,
7− "document_hash": "876f090ca1801217ec4295964e7d9ce5b46b9fc96b04db7ca463d6cc8ed74978", 7+ "document_hash": "065b6c18bbd16f16126924a54946ca4bf95c5863e57b14f33e4c9d88568f8549",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00202_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00202_2000", 2 "document_id": "exec_00202_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 68.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 71.74, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 31.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 43.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 5, \"unit_price\": 75.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 76.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 68.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 71.74, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 31.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 43.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 5, \"unit_price\": 75.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 76.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 7844, 6+ "char_len": 7665,
7− "document_hash": "660fc77600d7958889cef9b34f9c9a1a8c2d96ef38db054d180f225ebe3d4a42", 7+ "document_hash": "ec13194237af7efe7765a21f1d8303ab8ac449271eb48e3395906648c770bab8",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00203_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00203_2000", 2 "document_id": "exec_00203_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 73.68, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 40.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 27.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 78.27, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 49.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 53.29, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 73.68, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 40.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 27.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 78.27, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 49.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 53.29, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6− "char_len": 7967, 6+ "char_len": 7788,
7− "document_hash": "f26e049133c3f806d2045fa255ba4e6c2594f5ecbe872414b75faaf52ceb5033", 7+ "document_hash": "39eee37c7ba5fe1b6d8fce9eef4c00e10a33d8ca049d12ae01b9521d4ac60863",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00204_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00204_2000", 2 "document_id": "exec_00204_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 22.64, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 40, \"unit_price\": 49.16, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 47.27, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 36, \"unit_price\": 38.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 29.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 22.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 22.64, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 40, \"unit_price\": 49.16, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 47.27, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 36, \"unit_price\": 38.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 29.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 22.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 8100, 6+ "char_len": 7921,
7− "document_hash": "7cc296695d97fdf8fb07de5772744af9c3a79a52f20ee1e3410187cac4dd94b2", 7+ "document_hash": "0aaf1a095160aa21c4485db3b1b171e86d9d41b904ac2a2ea102cc13df19cd22",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00205_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00205_2000", 2 "document_id": "exec_00205_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 21.72, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 33.96, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 7, \"unit_price\": 20.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 24.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 29.74, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 1, \"unit_price\": 9.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 21.72, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 33.96, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 7, \"unit_price\": 20.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 24.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 29.74, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 1, \"unit_price\": 9.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 8061, 6+ "char_len": 7882,
7− "document_hash": "1f9b534774ca095126415c70bc29f4400d55f083ec020a658e6ee1263a76072f", 7+ "document_hash": "c82f149ddbd77be80773d103825621fb187f06bb60d4fd570c73c2b3cd898ecb",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00206_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00206_2000", 2 "document_id": "exec_00206_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 41.55, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 10.75, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 79.56, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 44.39, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 12.61, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 35.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 41.55, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 10.75, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 79.56, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 44.39, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 12.61, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 35.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6− "char_len": 7908, 6+ "char_len": 7727,
7− "document_hash": "71e5bf98eb246c53babd99cdd8ec1212e594a1af3ddd2a6da69bac721ad65ddd", 7+ "document_hash": "e391c9d757bc03677c664222c3ff3f1bbab881d7f566c0072b184e0b90503339",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00207_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00207_2000", 2 "document_id": "exec_00207_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 61.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 57.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 30.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 25.39, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 54.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 66.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 61.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 57.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 30.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 25.39, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 54.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 66.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6− "char_len": 8035, 6+ "char_len": 7856,
7− "document_hash": "2273befa734c54f0d780db1126ec3ce69ce8f9f829e57586a614786258681ba3", 7+ "document_hash": "7d024469b5d38776a899aa870f5af7e146953a19f1adc1d91f634cda62aa4c14",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00208_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00208_2000", 2 "document_id": "exec_00208_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 34.08, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 10, \"unit_price\": 62.88, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 63.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 23.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 78.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 32, \"unit_price\": 71.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 34.08, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 10, \"unit_price\": 62.88, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 63.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 23.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 78.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 32, \"unit_price\": 71.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 8036, 6+ "char_len": 7854,
7− "document_hash": "e22e990c3c20d7feee2f70e993961e465ac68d3a867c3547dc0566739847c2d8", 7+ "document_hash": "8528b3fafcf58b9daa88ffa9d7b9b25fb6e225f9bb767849744c05510a5ace37",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00209_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00209_2000", 2 "document_id": "exec_00209_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 42.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 72.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 23.31, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 63.77, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 32.46, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 52.71, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 42.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 72.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 23.31, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 63.77, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 32.46, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 52.71, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8044, 6+ "char_len": 7865,
7− "document_hash": "bd895760d266f605848302775beaceb129633108ddc74dcbe83a8642eecb0ea3", 7+ "document_hash": "27a0f4409b3611173adf72865b52c6beef014bce0282fd0498be50deeed192c9",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00210_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00210_2000", 2 "document_id": "exec_00210_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 64.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 76.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 77.13, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 33.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 77.57, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 19.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 64.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 76.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 77.13, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 33.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 77.57, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 19.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7990, 6+ "char_len": 7811,
7− "document_hash": "2678fb8184c3c22ce8bfa861af1a68ef2704dd84cb88718d63c1b668adaf1f69", 7+ "document_hash": "ffa82307f9b4986f3fc473b8b7df1e4f8f0e1c25d3b6e36cdcf4114a763427c5",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00211_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00211_2000", 2 "document_id": "exec_00211_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 44.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 18.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 19.62, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 61.32, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 64.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 15.18, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 44.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 18.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 19.62, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 61.32, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 64.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 15.18, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 8091, 6+ "char_len": 7909,
7− "document_hash": "5ad4fd06e4149025eddafab6fbf5c707e06c1f7e47f3cbaec839f52cd6744e38", 7+ "document_hash": "95faa51b0302db54273e9b4469b6eae92c251463238fdad494d437435c293548",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00212_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00212_2000", 2 "document_id": "exec_00212_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 38.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 5.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 68.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 31.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 68.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 17.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 38.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 5.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 68.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 31.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 68.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 17.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8061, 6+ "char_len": 7882,
7− "document_hash": "8a171bf76e4dc9c8921e9b2a4370fc5385ceb9695d62d19fc693110c84df06f0", 7+ "document_hash": "2be5dc8496484a99fc11a00be474f4f7f139a8f491a2ebf3fff0db8e7804dd82",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00213_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00213_2000", 2 "document_id": "exec_00213_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 6.93, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 72.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 79.31, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 20.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 16.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 59.43, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 6.93, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 72.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 79.31, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 20.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 16.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 59.43, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 7876, 6+ "char_len": 7695,
7− "document_hash": "55fc33fe81ccfe25e3ebdcdb70ee135b990fcf94d02bfb56e0116982ffd6df6f", 7+ "document_hash": "3c65c42333c25f1f3f90331747f08fafce263c8faa11a56d3a68bc58b2217327",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00214_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00214_2000", 2 "document_id": "exec_00214_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 62.71, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 20, \"unit_price\": 4.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 37.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 54.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 24.23, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 14, \"unit_price\": 46.97, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 62.71, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 20, \"unit_price\": 4.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 37.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 54.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 24.23, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 14, \"unit_price\": 46.97, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 7997, 6+ "char_len": 7815,
7− "document_hash": "0be81f6d80f5eb03ba862cbc94c8bf20b6ab1196483c947401585e7178fd458d", 7+ "document_hash": "ac0a3ee69ec7da626616f6ffb5e443a4f26a3d7857caaa20f1d4ebfa5355d28b",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00215_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00215_2000", 2 "document_id": "exec_00215_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 33.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 2.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 34.09, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 66.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 32.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 39.54, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 33.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 2.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 34.09, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 66.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 32.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 39.54, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6− "char_len": 8073, 6+ "char_len": 7894,
7− "document_hash": "37b95a047621c2a78beaff095b27647064c994bf6ede6ef6b3e91f8296b70eff", 7+ "document_hash": "a4094c70181359b8442ead384be9497e3fb0c46a4d9266cce07ca77bd9e26e2b",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00216_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00216_2000", 2 "document_id": "exec_00216_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 19.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 57.35, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 56.48, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 70.24, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 66.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 33.67, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 19.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 57.35, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 56.48, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 70.24, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 66.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 33.67, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8017, 6+ "char_len": 7838,
7− "document_hash": "1deddf301b0b2291cb9bdc4213848eceb17ece12032a2e454bcceb4ca4174397", 7+ "document_hash": "1a218de32e6fe4bbe97927ea16330bda3b598ed66c1517824d94ff90f944e802",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00217_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00217_2000", 2 "document_id": "exec_00217_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 71.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 23.77, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 22, \"unit_price\": 39.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 40.72, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 36.45, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 51.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 71.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 23.77, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 22, \"unit_price\": 39.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 40.72, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 36.45, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 51.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 7946, 6+ "char_len": 7764,
7− "document_hash": "7f71cb12d994e6fe2646a0f7b337c441f923868ec6b21c29533122fda4d87a73", 7+ "document_hash": "1223c7391b1da1aeae2d242c0e41de4639218cfe0c90ba9065f1cce42f0eedbd",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00218_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00218_2000", 2 "document_id": "exec_00218_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 69.57, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 11, \"unit_price\": 53.59, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 77.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 24, \"unit_price\": 22.35, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 79.4, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 20.83, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 69.57, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 11, \"unit_price\": 53.59, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 77.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 24, \"unit_price\": 22.35, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 79.4, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 20.83, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 7955, 6+ "char_len": 7776,
7− "document_hash": "2a77fd38142317b7e8868dfcaebb42248ddd462e9339268378a26e4b587eab3d", 7+ "document_hash": "971d94c4e7b4ae02142ffb6eb92af9d2cc0f6e4d52e94a64c5132b510197ce8a",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00219_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00219_2000", 2 "document_id": "exec_00219_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 79.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 39, \"unit_price\": 77.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 27.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 6.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 50.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 13.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 79.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 39, \"unit_price\": 77.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 27.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 6.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 50.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 13.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8119, 6+ "char_len": 7938,
7− "document_hash": "501d502ac1cd6920ddaeccbc4f0c925ddf4ca966cfb601ffd9d0547d97a741ed", 7+ "document_hash": "390edd848add1a4cf011660e8ac9a6a6aa381c4508a8f68635f3217ce20979cb",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00220_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00220_2000", 2 "document_id": "exec_00220_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 67.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 37.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 44.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 27.9, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 7.8, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 61.13, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 67.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 37.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 44.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 27.9, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 7.8, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 61.13, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7931, 6+ "char_len": 7750,
7− "document_hash": "a7487c2eb12039fcc44c64460641febdd19eaf0e13d8b6b8f00d018ca564a8b6", 7+ "document_hash": "31eca9e7afc70ac5a795edc1d1529c531e0d028342802a61f4496c013e13bf45",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00221_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00221_2000", 2 "document_id": "exec_00221_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 67.75, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 38.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 52.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 29.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 7.64, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 39, \"unit_price\": 27.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 67.75, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 38.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 52.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 29.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 7.64, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 39, \"unit_price\": 27.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 7820, 6+ "char_len": 7638,
7− "document_hash": "a986eec8e76aa658fd694272657b8e201d940be0671e7769a934ffe76ca4748a", 7+ "document_hash": "e228dfaa0d04173ee28b43513e5fc06914edf5d0c02b09bf0fc534531a583669",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00222_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00222_2000", 2 "document_id": "exec_00222_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 13.55, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 25.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 19.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 79.51, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 42.07, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 57.83, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 9}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 13.55, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 25.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 19.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 79.51, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 42.07, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 57.83, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 9}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8057, 6+ "char_len": 7878,
7− "document_hash": "fa31c738b8b80170524adedf048965eb739a7ebf7de9fcd4f4192d331dcc3346", 7+ "document_hash": "040260f25a42680fcc038386368506adaa004bc2592e75e3fbb5903b54cc19ab",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00223_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00223_2000", 2 "document_id": "exec_00223_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 19.44, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 15.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 51.74, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 11.86, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 33.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 44.89, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 19.44, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 15.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 51.74, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 11.86, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 33.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 44.89, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 7992, 6+ "char_len": 7813,
7− "document_hash": "9b2d718a88e7b451a6e6d433e81fa6c3862777588f40791fe888b2d7a44efa42", 7+ "document_hash": "7eef13486249d89cf05bda367ea9dae2ce2479b2236c1917b2f067c49a981619",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00224_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00224_2000", 2 "document_id": "exec_00224_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 40.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 74.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 67.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 44.81, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 79.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 60.0, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 40.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 74.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 67.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 44.81, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 79.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 60.0, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 7921, 6+ "char_len": 7739,
7− "document_hash": "526a53c484a99dde8d83b8fc83d103060128d6331659920d7688b8836de1ec53", 7+ "document_hash": "3e642abb57841365b827053f2437a250dcb419539389820bac778d4596c064fd",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00225_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00225_2000", 2 "document_id": "exec_00225_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 5.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 25.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 22.38, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 3, \"unit_price\": 5.01, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 70.26, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 5.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 5.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 25.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 22.38, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 3, \"unit_price\": 5.01, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 70.26, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 5.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 7998, 6+ "char_len": 7819,
7− "document_hash": "0d5d08bcef45a939605c15cb0d0b5b1ac3c2304c8358dae05fa7da75db07621f", 7+ "document_hash": "54918c16a4ae3706a87d182112fca2a39c82c77e3a60fd25a8b47b958f2a6b21",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00226_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00226_2000", 2 "document_id": "exec_00226_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 13.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 77.55, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 40.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 65.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 43.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 50.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 13.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 77.55, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 40.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 65.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 43.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 50.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7972, 6+ "char_len": 7793,
7− "document_hash": "32ca9297d301e1956917680ad370eda2add0304f9742c49a2023cfa3f2a82bd2", 7+ "document_hash": "7a73067df93b17aba1a608bce48f7a6ff16d480af8d23713ec889405f279fc7d",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00227_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00227_2000", 2 "document_id": "exec_00227_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 60.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 16, \"unit_price\": 74.12, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 66.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 49.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 14.6, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 54.41, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 60.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 16, \"unit_price\": 74.12, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 66.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 49.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 14.6, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 54.41, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6− "char_len": 8074, 6+ "char_len": 7895,
7− "document_hash": "0297d0253d8ecb7ba91156d4221aff1d54ba5efaed7fe7664734e63d44d31bd0", 7+ "document_hash": "6afe19e10d57be0082db797187045b004680fa22a13a56a7b8f7e2a51b40597d",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00228_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00228_2000", 2 "document_id": "exec_00228_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 76.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 29, \"unit_price\": 33.94, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 14, \"unit_price\": 14.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 4.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 52.45, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 77.48, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 76.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 29, \"unit_price\": 33.94, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 14, \"unit_price\": 14.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 4.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 52.45, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 77.48, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7994, 6+ "char_len": 7815,
7− "document_hash": "7bd8eb7ac7409f89d5f44f1f5001fd0660aece03d12d92588e821c86e0d1f69b", 7+ "document_hash": "08d5c40fa24cccbcccaac14d408c241ab602604a712b727f12f570eb56887a74",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00229_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00229_2000", 2 "document_id": "exec_00229_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 76.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 78.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 41.6, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 3.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 31.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 76.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 78.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 41.6, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 3.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 31.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 7905, 6+ "char_len": 7723,
7− "document_hash": "488c5efd60f8e64b88269226071b8befe058a5e2442b99934157b38f5fc41a19", 7+ "document_hash": "b6a34443a51bb95703036fc2467d0c50ad248f99f3c11c0a35d6bc7fcf0ad481",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00230_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00230_2000", 2 "document_id": "exec_00230_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 34.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 36.88, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 42.84, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 25.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 20.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 69.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 34.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 36.88, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 42.84, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 25.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 20.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 69.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6− "char_len": 8012, 6+ "char_len": 7830,
7− "document_hash": "2e09e5c09991cb1b7c5c288b9a49bb058042a17dc7fd32cde98e2b705477598c", 7+ "document_hash": "cbb9dba4a75d2570390205536d928042eae0341c49e7dcc11a2cea988c175ba9",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00231_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00231_2000", 2 "document_id": "exec_00231_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 34, \"unit_price\": 42.27, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 66.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 77.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 28, \"unit_price\": 73.35, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 48.71, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 4.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 34, \"unit_price\": 42.27, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 66.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 77.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 28, \"unit_price\": 73.35, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 48.71, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 4.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8077, 6+ "char_len": 7895,
7− "document_hash": "9071b939e284c02834a84c70b2af1783b2a26d6217aa2184b3224f0c27400eaf", 7+ "document_hash": "b599ec4f6ea744baef17b36d05ab828f105675eeebf81a780befd49f389ec9d3",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00232_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00232_2000", 2 "document_id": "exec_00232_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 25.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 75.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 65.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 7, \"unit_price\": 59.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 11, \"unit_price\": 48.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 30.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 25.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 75.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 65.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 7, \"unit_price\": 59.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 11, \"unit_price\": 48.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 30.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7990, 6+ "char_len": 7811,
7− "document_hash": "6bb55261f58b559a6304971baddf54cabc7de9843d30afe350a659e21fa02b73", 7+ "document_hash": "94e54bf28dac5a404fdf94451d3e632c18f63dff001cc758f89f5239d27f659b",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00233_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00233_2000", 2 "document_id": "exec_00233_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 40.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 16, \"unit_price\": 13.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 1, \"unit_price\": 30.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 77.63, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 21.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 7.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 40.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 16, \"unit_price\": 13.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 1, \"unit_price\": 30.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 77.63, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 21.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 7.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6− "char_len": 7909, 6+ "char_len": 7730,
7− "document_hash": "f88eb8610ee0ae9a0cc4c953e6f302bceb7e6a9f131c2e5f775a4026d048e997", 7+ "document_hash": "c0f56e7b9b73df623955c8b6822884e4b53df031c58736213f498ed2b4d693dc",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00234_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00234_2000", 2 "document_id": "exec_00234_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 43.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 31.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 28, \"unit_price\": 16.65, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 40.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 33.7, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 43.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 31.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 28, \"unit_price\": 16.65, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 40.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 33.7, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8006, 6+ "char_len": 7827,
7− "document_hash": "89b49129e4581b4b424c4986c040ee62a62ff6f482d53151d96cbcf944c97e02", 7+ "document_hash": "3522724b66617b60e55fa67efc23998ff528130352e7a960ee1e052d697670a5",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00235_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00235_2000", 2 "document_id": "exec_00235_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 21, \"unit_price\": 57.5, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 43.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 70.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 13, \"unit_price\": 4.78, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 3.2, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 1, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 21, \"unit_price\": 57.5, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 43.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 70.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 13, \"unit_price\": 4.78, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 3.2, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 1, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6− "char_len": 8079, 6+ "char_len": 7900,
7− "document_hash": "7870795b3e6c98bee59eb42d0e036ccc1d88fe696a6e7c94016c77fb0f3cefe4", 7+ "document_hash": "20bfca2f1ed7277720116b37988eb2bf9331505c61f7491ff7f131ed0b2d70e1",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00236_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00236_2000", 2 "document_id": "exec_00236_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 35.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 13.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 55.74, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 26.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 20.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 33.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 35.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 13.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 55.74, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 26.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 20.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 33.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6− "char_len": 8048, 6+ "char_len": 7867,
7− "document_hash": "dabe94079fad1aa1fef7653ff5583522be249aa0c33c58334ae1d261091d83f9", 7+ "document_hash": "23053422d45849043d9eb3706daf105a581cd66b69e96220ab1285c7c92b2058",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00237_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00237_2000", 2 "document_id": "exec_00237_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.21, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 78.67, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 39.59, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 23.76, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 26.17, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 31.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.21, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 78.67, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 39.59, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 23.76, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 26.17, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 31.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6− "char_len": 8024, 6+ "char_len": 7842,
7− "document_hash": "455d86e6a8f1e9be0eaebb2e2ab279c0439c6768273a410e7994bc6a76dfbecf", 7+ "document_hash": "a8320d4e9d153e4398af56ca92c73f39bb1c1b9e97609e9ec096c1157cebe872",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00238_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00238_2000", 2 "document_id": "exec_00238_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 46.52, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 14.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 18.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 15.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 46.52, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 14.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 18.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 15.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6− "char_len": 7855, 6+ "char_len": 7676,
7− "document_hash": "ffdd32b5c64c4cab201265cf785b962e0e60f2f65cf46cfd2674256d0a97fe14", 7+ "document_hash": "068a11a09e40cb07626e21d7abb095fb9c4779acc1a0db602ca10678ae79e3e0",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00239_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00239_2000", 2 "document_id": "exec_00239_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 51.01, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 47.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 14.87, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 68.34, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 15.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 10.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 51.01, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 47.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 14.87, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 68.34, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 15.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 10.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7905, 6+ "char_len": 7723,
7− "document_hash": "1215eafbd18b1d396d2779e2c0227bfcff05e59f1f06d602fc8284f18da6bbc8", 7+ "document_hash": "015c78e1e480fb805976a26912f0308a79a0c4b2122313463b5ff6b5ee996a9b",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00240_2000.json +3 −3
@@ -2,9 +2,9 @@
2 "document_id": "exec_00240_2000", 2 "document_id": "exec_00240_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "text": "ENGINEERING TICKET\n\nBug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 38.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 28.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 23, \"unit_price\": 35.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 75.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.6, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 35, \"unit_price\": 18.22, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.", 5+ "text": "ENGINEERING TICKET\n\nBug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 38.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 28.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 23, \"unit_price\": 35.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 75.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.6, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 35, \"unit_price\": 18.22, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6− "char_len": 7976, 6+ "char_len": 7797,
7− "document_hash": "2598ae3b4041685f7fbf0de4eb4c12a5508a3c374ae89cdf7007d3cf7e6e5924", 7+ "document_hash": "4c3927b8378a2b949e85371101d959a0adebcbe41a2d39f239aae883de382e19",
8 "questions": [ 8 "questions": [
9 { 9 {
10 "question_id": "q00_exec", 10 "question_id": "q00_exec",
modified data/generated/keys/exec/exec_00001_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00001_2000", 2 "document_id": "exec_00001_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "ddfe7fcc3baca3e97144b66188f285a0e8a83018d1ca41b7f37cb80dede62dcc", 5+ "document_hash": "72a702cc7076fb313f97772498675d83532bf6ce59f1d3cbf3bb86fa387dc0e6",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 751 30+ "char_end": 624
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 7 124 "shipped_days": 7
125 } 125 }
126 ], 126 ],
127− "expected": 3629.0 127+ "expected": 3711.0
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -269,7 +269,7 @@
269 "shipped_days": 7 269 "shipped_days": 7
270 } 270 }
271 ], 271 ],
272− "expected": 4601.0 272+ "expected": 4683.0
273 }, 273 },
274 { 274 {
275 "input": [ 275 "input": [
@@ -400,7 +400,7 @@
400 "shipped_days": 5 400 "shipped_days": 5
401 } 401 }
402 ], 402 ],
403− "expected": 4660.0 403+ "expected": 4778.0
404 }, 404 },
405 { 405 {
406 "input": [ 406 "input": [
@@ -459,7 +459,7 @@
459 "shipped_days": 2 459 "shipped_days": 2
460 } 460 }
461 ], 461 ],
462− "expected": 2896.0 462+ "expected": 2962.0
463 } 463 }
464 ] 464 ]
465 } 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00002_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00002_2000", 2 "document_id": "exec_00002_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "7918e3cdcab2a8bfae77558b7b523f4cea70cdc52435393d98d0c7b36de29e60", 5+ "document_hash": "7ec85eb17fda1c57c2a77a31dc79bf35b3c62974f6f1905e409880cbe869b6f5",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 3 124 "shipped_days": 3
125 } 125 }
126 ], 126 ],
127− "expected": 2969.0 127+ "expected": 3084.0
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -165,7 +165,7 @@
165 "shipped_days": 11 165 "shipped_days": 11
166 } 166 }
167 ], 167 ],
168− "expected": 752.0 168+ "expected": 794.0
169 }, 169 },
170 { 170 {
171 "input": [ 171 "input": [
@@ -233,7 +233,7 @@
233 "shipped_days": 11 233 "shipped_days": 11
234 } 234 }
235 ], 235 ],
236− "expected": 2144.0 236+ "expected": 2258.0
237 }, 237 },
238 { 238 {
239 "input": [ 239 "input": [
@@ -391,7 +391,7 @@
391 "shipped_days": 5 391 "shipped_days": 5
392 } 392 }
393 ], 393 ],
394− "expected": 8977.0 394+ "expected": 9175.0
395 }, 395 },
396 { 396 {
397 "input": [ 397 "input": [
@@ -531,7 +531,7 @@
531 "shipped_days": 5 531 "shipped_days": 5
532 } 532 }
533 ], 533 ],
534− "expected": 7566.0 534+ "expected": 7764.0
535 } 535 }
536 ] 536 ]
537 } 537 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00003_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00003_2000", 2 "document_id": "exec_00003_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "8ab845867bddfdcb5cf63b1319a6f29794cc9dddea45cd92bca3bc94fe023da7", 5+ "document_hash": "f7d729b32469d142625681408a1dd1aa3d1d99a7c07adb645c9f40d7550e8695",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 "shipped_days": 13 115 "shipped_days": 13
116 } 116 }
117 ], 117 ],
118− "expected": 3222.6 118+ "expected": 3349.8
119 }, 119 },
120 { 120 {
121 "input": [ 121 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 13 228 "shipped_days": 13
229 } 229 }
230 ], 230 ],
231− "expected": 6782.7 231+ "expected": 7004.7
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -269,7 +269,7 @@
269 "shipped_days": 13 269 "shipped_days": 13
270 } 270 }
271 ], 271 ],
272− "expected": 1236.8 272+ "expected": 1248.8
273 }, 273 },
274 { 274 {
275 "input": [ 275 "input": [
@@ -391,7 +391,7 @@
391 "shipped_days": 13 391 "shipped_days": 13
392 } 392 }
393 ], 393 ],
394− "expected": 7525.7 394+ "expected": 7800.7
395 }, 395 },
396 { 396 {
397 "input": [ 397 "input": [
@@ -486,7 +486,7 @@
486 "shipped_days": 14 486 "shipped_days": 14
487 } 487 }
488 ], 488 ],
489− "expected": 6066.7 489+ "expected": 6309.5
490 } 490 }
491 ] 491 ]
492 } 492 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00004_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00004_2000", 2 "document_id": "exec_00004_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "08c5cdfaa7a0742a094ee4726d97a32a89395c354270406bbcf78eff4fdbe8b2", 5+ "document_hash": "aeca575de77be2cf25f1c9f8e053485ad918ef4b3468680a4ef6bfa3b8cb781b",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 3, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 761 30+ "char_end": 634
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 "shipped_days": 9 97 "shipped_days": 9
98 } 98 }
99 ], 99 ],
100− "expected": 5428.38 100+ "expected": 5736.55
101 }, 101 },
102 { 102 {
103 "input": [ 103 "input": [
@@ -147,7 +147,7 @@
147 "shipped_days": 2 147 "shipped_days": 2
148 } 148 }
149 ], 149 ],
150− "expected": 6465.92 150+ "expected": 6671.26
151 }, 151 },
152 { 152 {
153 "input": [ 153 "input": [
@@ -251,7 +251,7 @@
251 "shipped_days": 9 251 "shipped_days": 9
252 } 252 }
253 ], 253 ],
254− "expected": 10962.43 254+ "expected": 11280.56
255 }, 255 },
256 { 256 {
257 "input": [ 257 "input": [
@@ -310,7 +310,7 @@
310 "shipped_days": 2 310 "shipped_days": 2
311 } 311 }
312 ], 312 ],
313− "expected": 5425.82 313+ "expected": 5570.86
314 }, 314 },
315 { 315 {
316 "input": [ 316 "input": [
@@ -441,7 +441,7 @@
441 "shipped_days": 12 441 "shipped_days": 12
442 } 442 }
443 ], 443 ],
444− "expected": 9991.84 444+ "expected": 10440.76
445 } 445 }
446 ] 446 ]
447 } 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00005_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00005_2000", 2 "document_id": "exec_00005_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "f885d791d02ed1adb010c9c27804b5c38facf8fd2a09b9b1b6f9cacb7b0acc81", 5+ "document_hash": "c5e26772985f1897f27e3729a56bdf95e08ba4718b928a1480572409c422c83f",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 751 30+ "char_end": 624
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -196,7 +196,7 @@
196 "shipped_days": 5 196 "shipped_days": 5
197 } 197 }
198 ], 198 ],
199− "expected": 10831.0 199+ "expected": 11286.0
200 }, 200 },
201 { 201 {
202 "input": [ 202 "input": [
@@ -354,7 +354,7 @@
354 "shipped_days": 11 354 "shipped_days": 11
355 } 355 }
356 ], 356 ],
357− "expected": 9939.0 357+ "expected": 10311.0
358 }, 358 },
359 { 359 {
360 "input": [ 360 "input": [
@@ -413,7 +413,7 @@
413 "shipped_days": 8 413 "shipped_days": 8
414 } 414 }
415 ], 415 ],
416− "expected": 4721.0 416+ "expected": 4918.0
417 }, 417 },
418 { 418 {
419 "input": [ 419 "input": [
@@ -598,7 +598,7 @@
598 "shipped_days": 10 598 "shipped_days": 10
599 } 599 }
600 ], 600 ],
601− "expected": 15203.0 601+ "expected": 15877.0
602 }, 602 },
603 { 603 {
604 "input": [ 604 "input": [
@@ -675,7 +675,7 @@
675 "shipped_days": 5 675 "shipped_days": 5
676 } 676 }
677 ], 677 ],
678− "expected": 4292.0 678+ "expected": 4542.0
679 } 679 }
680 ] 680 ]
681 } 681 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00006_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00006_2000", 2 "document_id": "exec_00006_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "2a1bb6d86c77861b166e281bb1deac9a658ed4383a7ade93e9698e33861de0e5", 5+ "document_hash": "283461ef1dfaf2225620026d081a07116ec5fa2173ffa118704c0cf1d53a2eac",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 758 30+ "char_end": 631
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 "shipped_days": 4 169 "shipped_days": 4
170 } 170 }
171 ], 171 ],
172− "expected": 13971.0 172+ "expected": 14483.0
173 }, 173 },
174 { 174 {
175 "input": [ 175 "input": [
@@ -291,7 +291,7 @@
291 "shipped_days": 3 291 "shipped_days": 3
292 } 292 }
293 ], 293 ],
294− "expected": 13520.0 294+ "expected": 14032.0
295 }, 295 },
296 { 296 {
297 "input": [ 297 "input": [
@@ -422,7 +422,7 @@
422 "shipped_days": 10 422 "shipped_days": 10
423 } 423 }
424 ], 424 ],
425− "expected": 11047.0 425+ "expected": 11404.0
426 }, 426 },
427 { 427 {
428 "input": [ 428 "input": [
@@ -526,7 +526,7 @@
526 "shipped_days": 13 526 "shipped_days": 13
527 } 527 }
528 ], 528 ],
529− "expected": 10526.0 529+ "expected": 11032.0
530 }, 530 },
531 { 531 {
532 "input": [ 532 "input": [
@@ -621,7 +621,7 @@
621 "shipped_days": 4 621 "shipped_days": 4
622 } 622 }
623 ], 623 ],
624− "expected": 9516.0 624+ "expected": 9886.0
625 } 625 }
626 ] 626 ]
627 } 627 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00007_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00007_2000", 2 "document_id": "exec_00007_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "fb0150ce17a4b0ade1601cf6082b82e83f67db83a57c177064777d028b33d97c", 5+ "document_hash": "961b89accd7b0332a165c291bb3c527fd00bea183807cff965326b5f758f1b7f",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 759 30+ "char_end": 632
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 "shipped_days": 8 160 "shipped_days": 8
161 } 161 }
162 ], 162 ],
163− "expected": 5439.03 163+ "expected": 5689.66
164 }, 164 },
165 { 165 {
166 "input": [ 166 "input": [
@@ -201,7 +201,7 @@
201 "shipped_days": 11 201 "shipped_days": 11
202 } 202 }
203 ], 203 ],
204− "expected": 2512.96 204+ "expected": 2661.57
205 }, 205 },
206 { 206 {
207 "input": [ 207 "input": [
@@ -260,7 +260,7 @@
260 "shipped_days": 8 260 "shipped_days": 8
261 } 261 }
262 ], 262 ],
263− "expected": 1312.49 263+ "expected": 1389.42
264 }, 264 },
265 { 265 {
266 "input": [ 266 "input": [
@@ -391,7 +391,7 @@
391 "shipped_days": 1 391 "shipped_days": 1
392 } 392 }
393 ], 393 ],
394− "expected": 6947.25 394+ "expected": 7280.02
395 }, 395 },
396 { 396 {
397 "input": [ 397 "input": [
@@ -441,7 +441,7 @@
441 "shipped_days": 8 441 "shipped_days": 8
442 } 442 }
443 ], 443 ],
444− "expected": 3350.53 444+ "expected": 3547.57
445 } 445 }
446 ] 446 ]
447 } 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00008_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00008_2000", 2 "document_id": "exec_00008_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "04f896fa681f3ae7cbfc7efa30fd96c43be9d9b703dab76457ee8043483b3612", 5+ "document_hash": "4ad9a1fbe7db8f523841d367742505a5c92581010a15fdabb2bb6b6d77afbaa4",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 5 133 "shipped_days": 5
134 } 134 }
135 ], 135 ],
136− "expected": 6146.0 136+ "expected": 6369.0
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -264,7 +264,7 @@
264 "shipped_days": 6 264 "shipped_days": 6
265 } 265 }
266 ], 266 ],
267− "expected": 7379.0 267+ "expected": 7689.0
268 }, 268 },
269 { 269 {
270 "input": [ 270 "input": [
@@ -323,7 +323,7 @@
323 "shipped_days": 14 323 "shipped_days": 14
324 } 324 }
325 ], 325 ],
326− "expected": 4608.0 326+ "expected": 4853.0
327 }, 327 },
328 { 328 {
329 "input": [ 329 "input": [
@@ -382,7 +382,7 @@
382 "shipped_days": 1 382 "shipped_days": 1
383 } 383 }
384 ], 384 ],
385− "expected": 3065.0 385+ "expected": 3218.0
386 }, 386 },
387 { 387 {
388 "input": [ 388 "input": [
@@ -513,7 +513,7 @@
513 "shipped_days": 2 513 "shipped_days": 2
514 } 514 }
515 ], 515 ],
516− "expected": 7379.0 516+ "expected": 7689.0
517 } 517 }
518 ] 518 ]
519 } 519 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00009_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00009_2000", 2 "document_id": "exec_00009_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "db44cbab6b33239dddaa205976960167be792c1b1025bd39a4c4d6cf0f6ab7d7", 5+ "document_hash": "7c7609e3e3780713446d7ec4cbe48dba3ad69a1bf6d96393eb7e34fe13581d81",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 1 133 "shipped_days": 1
134 } 134 }
135 ], 135 ],
136− "expected": 6224.0 136+ "expected": 6375.0
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -273,7 +273,7 @@
273 "shipped_days": 1 273 "shipped_days": 1
274 } 274 }
275 ], 275 ],
276− "expected": 8620.0 276+ "expected": 8850.0
277 }, 277 },
278 { 278 {
279 "input": [ 279 "input": [
@@ -368,7 +368,7 @@
368 "shipped_days": 14 368 "shipped_days": 14
369 } 369 }
370 ], 370 ],
371− "expected": 3550.0 371+ "expected": 3637.0
372 }, 372 },
373 { 373 {
374 "input": [ 374 "input": [
@@ -508,7 +508,7 @@
508 "shipped_days": 8 508 "shipped_days": 8
509 } 509 }
510 ], 510 ],
511− "expected": 7854.0 511+ "expected": 8104.0
512 }, 512 },
513 { 513 {
514 "input": [ 514 "input": [
@@ -585,7 +585,7 @@
585 "shipped_days": 8 585 "shipped_days": 8
586 } 586 }
587 ], 587 ],
588− "expected": 5171.0 588+ "expected": 5396.0
589 } 589 }
590 ] 590 ]
591 } 591 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00010_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00010_2000", 2 "document_id": "exec_00010_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "232fe52471a905a3ae14b591781c17dfa57a2d115258bfc1f8a1fce231b705ad", 5+ "document_hash": "2951e556f1045edd4bb9926de1dac81793260ddcd823e8c6abe38566b4cbcc07",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 751 30+ "char_end": 624
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 11 124 "shipped_days": 11
125 } 125 }
126 ], 126 ],
127− "expected": 2617.0 127+ "expected": 2715.0
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -183,7 +183,7 @@
183 "shipped_days": 6 183 "shipped_days": 6
184 } 184 }
185 ], 185 ],
186− "expected": 1214.0 186+ "expected": 1237.0
187 }, 187 },
188 { 188 {
189 "input": [ 189 "input": [
@@ -323,7 +323,7 @@
323 "shipped_days": 7 323 "shipped_days": 7
324 } 324 }
325 ], 325 ],
326− "expected": 3768.0 326+ "expected": 3907.0
327 }, 327 },
328 { 328 {
329 "input": [ 329 "input": [
@@ -382,7 +382,7 @@
382 "shipped_days": 10 382 "shipped_days": 10
383 } 383 }
384 ], 384 ],
385− "expected": 694.0 385+ "expected": 724.0
386 }, 386 },
387 { 387 {
388 "input": [ 388 "input": [
@@ -567,7 +567,7 @@
567 "shipped_days": 11 567 "shipped_days": 11
568 } 568 }
569 ], 569 ],
570− "expected": 5560.0 570+ "expected": 5735.0
571 } 571 }
572 ] 572 ]
573 } 573 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00011_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00011_2000", 2 "document_id": "exec_00011_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "f046c8292307adf97cae067d41f5a7bcce9d2a48bc8dd143e0f11e89a1c3a4c2", 5+ "document_hash": "6b5b870bbcdf744c667890cf6185717b10f87e45ad7a6ccada501987d53f5e6d",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 753 30+ "char_end": 626
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 13 88 "shipped_days": 13
89 } 89 }
90 ], 90 ],
91− "expected": 3443.0 91+ "expected": 3644.0
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -192,7 +192,7 @@
192 "shipped_days": 13 192 "shipped_days": 13
193 } 193 }
194 ], 194 ],
195− "expected": 4208.0 195+ "expected": 4474.0
196 }, 196 },
197 { 197 {
198 "input": [ 198 "input": [
@@ -287,7 +287,7 @@
287 "shipped_days": 14 287 "shipped_days": 14
288 } 288 }
289 ], 289 ],
290− "expected": 3787.0 290+ "expected": 4024.0
291 }, 291 },
292 { 292 {
293 "input": [ 293 "input": [
@@ -418,7 +418,7 @@
418 "shipped_days": 7 418 "shipped_days": 7
419 } 419 }
420 ], 420 ],
421− "expected": 5822.0 421+ "expected": 6177.0
422 }, 422 },
423 { 423 {
424 "input": [ 424 "input": [
@@ -540,7 +540,7 @@
540 "shipped_days": 14 540 "shipped_days": 14
541 } 541 }
542 ], 542 ],
543− "expected": 5822.0 543+ "expected": 6177.0
544 } 544 }
545 ] 545 ]
546 } 546 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00012_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00012_2000", 2 "document_id": "exec_00012_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "2befcba5bfd5b509a9a6ec5d9334e2e384a05b6733ac02d80222302d3ca6e6ef", 5+ "document_hash": "8991dd153dfd83b4a1aea0eb4340aabe4370ac72ef16427f5f67894ac175edde",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -196,7 +196,7 @@
196 "shipped_days": 13 196 "shipped_days": 13
197 } 197 }
198 ], 198 ],
199− "expected": 6955.0 199+ "expected": 7281.0
200 }, 200 },
201 { 201 {
202 "input": [ 202 "input": [
@@ -237,7 +237,7 @@
237 "shipped_days": 10 237 "shipped_days": 10
238 } 238 }
239 ], 239 ],
240− "expected": 2047.0 240+ "expected": 2140.0
241 }, 241 },
242 { 242 {
243 "input": [ 243 "input": [
@@ -323,7 +323,7 @@
323 "shipped_days": 2 323 "shipped_days": 2
324 } 324 }
325 ], 325 ],
326− "expected": 4220.0 326+ "expected": 4380.0
327 }, 327 },
328 { 328 {
329 "input": [ 329 "input": [
@@ -490,7 +490,7 @@
490 "shipped_days": 8 490 "shipped_days": 8
491 } 491 }
492 ], 492 ],
493− "expected": 7371.0 493+ "expected": 7658.0
494 }, 494 },
495 { 495 {
496 "input": [ 496 "input": [
@@ -603,7 +603,7 @@
603 "shipped_days": 2 603 "shipped_days": 2
604 } 604 }
605 ], 605 ],
606− "expected": 3079.0 606+ "expected": 3117.0
607 } 607 }
608 ] 608 ]
609 } 609 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00013_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00013_2000", 2 "document_id": "exec_00013_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "d006207e385cff8703be29071faf5845de84d05d8a677e40dda4983739115d29", 5+ "document_hash": "ee93ee22294a4463cb6cf9ca3d5ab7ea7fd2a991b2b55448f089c2290e51a174",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 753 30+ "char_end": 626
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -151,7 +151,7 @@
151 "shipped_days": 2 151 "shipped_days": 2
152 } 152 }
153 ], 153 ],
154− "expected": 5458.0 154+ "expected": 5773.0
155 }, 155 },
156 { 156 {
157 "input": [ 157 "input": [
@@ -219,7 +219,7 @@
219 "shipped_days": 2 219 "shipped_days": 2
220 } 220 }
221 ], 221 ],
222− "expected": 1842.0 222+ "expected": 1944.0
223 }, 223 },
224 { 224 {
225 "input": [ 225 "input": [
@@ -296,7 +296,7 @@
296 "shipped_days": 8 296 "shipped_days": 8
297 } 297 }
298 ], 298 ],
299− "expected": 4882.0 299+ "expected": 5107.0
300 }, 300 },
301 { 301 {
302 "input": [ 302 "input": [
@@ -445,7 +445,7 @@
445 "shipped_days": 14 445 "shipped_days": 14
446 } 446 }
447 ], 447 ],
448− "expected": 7221.0 448+ "expected": 7577.0
449 }, 449 },
450 { 450 {
451 "input": [ 451 "input": [
@@ -504,7 +504,7 @@
504 "shipped_days": 9 504 "shipped_days": 9
505 } 505 }
506 ], 506 ],
507− "expected": 2757.0 507+ "expected": 2864.0
508 } 508 }
509 ] 509 ]
510 } 510 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00014_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00014_2000", 2 "document_id": "exec_00014_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "e6ac9234df3fc7dcec86770e041bf75b576d1c65cf6fe32717891429f5461066", 5+ "document_hash": "b25f9d4fb9c997b0cb6e19983b48490097f6592b8d17e66f305f5a1b3e8a8443",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 "shipped_days": 5 160 "shipped_days": 5
161 } 161 }
162 ], 162 ],
163− "expected": 8460.0 163+ "expected": 8930.0
164 }, 164 },
165 { 165 {
166 "input": [ 166 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 5 228 "shipped_days": 5
229 } 229 }
230 ], 230 ],
231− "expected": 2370.0 231+ "expected": 2505.0
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -305,7 +305,7 @@
305 "shipped_days": 4 305 "shipped_days": 4
306 } 306 }
307 ], 307 ],
308− "expected": 3607.0 308+ "expected": 3812.0
309 }, 309 },
310 { 310 {
311 "input": [ 311 "input": [
@@ -454,7 +454,7 @@
454 "shipped_days": 13 454 "shipped_days": 13
455 } 455 }
456 ], 456 ],
457− "expected": 11029.0 457+ "expected": 11639.0
458 }, 458 },
459 { 459 {
460 "input": [ 460 "input": [
@@ -558,7 +558,7 @@
558 "shipped_days": 4 558 "shipped_days": 4
559 } 559 }
560 ], 560 ],
561− "expected": 9013.0 561+ "expected": 9499.0
562 } 562 }
563 ] 563 ]
564 } 564 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00015_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00015_2000", 2 "document_id": "exec_00015_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "7bd733e903db470812271aa82f1cd9c7111ace0dcd257ace35b07e660d045be7", 5+ "document_hash": "7334bc5b260e11b2b0cdf9db05839cac2f397ae0e44db4e8bba7559dc94036bb",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 5, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 5, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 "shipped_days": 10 106 "shipped_days": 10
107 } 107 }
108 ], 108 ],
109− "expected": 2391.96 109+ "expected": 2525.61
110 }, 110 },
111 { 111 {
112 "input": [ 112 "input": [
@@ -219,7 +219,7 @@
219 "shipped_days": 10 219 "shipped_days": 10
220 } 220 }
221 ], 221 ],
222− "expected": 4322.59 222+ "expected": 4529.08
223 }, 223 },
224 { 224 {
225 "input": [ 225 "input": [
@@ -332,7 +332,7 @@
332 "shipped_days": 11 332 "shipped_days": 11
333 } 333 }
334 ], 334 ],
335− "expected": 2997.99 335+ "expected": 3122.6
336 }, 336 },
337 { 337 {
338 "input": [ 338 "input": [
@@ -454,7 +454,7 @@
454 "shipped_days": 1 454 "shipped_days": 1
455 } 455 }
456 ], 456 ],
457− "expected": 3013.36 457+ "expected": 3138.94
458 }, 458 },
459 { 459 {
460 "input": [ 460 "input": [
@@ -585,7 +585,7 @@
585 "shipped_days": 10 585 "shipped_days": 10
586 } 586 }
587 ], 587 ],
588− "expected": 4571.24 588+ "expected": 4788.45
589 } 589 }
590 ] 590 ]
591 } 591 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00016_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00016_2000", 2 "document_id": "exec_00016_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "b232fee620440e55facf033b892741387c97661b269dc2371313fb0f5934ff69", 5+ "document_hash": "6a5268852caa8dc3015fd42e678e912f463ba061ac417b09de9ac6ed45e85d7f",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 6, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 6, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 751 30+ "char_end": 624
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -187,7 +187,7 @@
187 "shipped_days": 12 187 "shipped_days": 12
188 } 188 }
189 ], 189 ],
190− "expected": 7227.8 190+ "expected": 7417.1
191 }, 191 },
192 { 192 {
193 "input": [ 193 "input": [
@@ -273,7 +273,7 @@
273 "shipped_days": 2 273 "shipped_days": 2
274 } 274 }
275 ], 275 ],
276− "expected": 4555.1 276+ "expected": 4666.4
277 }, 277 },
278 { 278 {
279 "input": [ 279 "input": [
@@ -364,7 +364,7 @@
364 "shipped_days": 6 364 "shipped_days": 6
365 } 365 }
366 ], 366 ],
367− "expected": 934.1 367+ "expected": 986.0
368 }, 368 },
369 { 369 {
370 "input": [ 370 "input": [
@@ -405,7 +405,7 @@
405 "shipped_days": 5 405 "shipped_days": 5
406 } 406 }
407 ], 407 ],
408− "expected": 2449.0 408+ "expected": 2494.6
409 } 409 }
410 ] 410 ]
411 } 411 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00017_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00017_2000", 2 "document_id": "exec_00017_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "bce532bd3702b32e6eb3620ef3ea7d860a695f22bb90b8e995836a7937c77f52", 5+ "document_hash": "f2d06422bd98b400b7401ce4c5ae2939aeb43f51482e84cba0c0e3c0845c71a5",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 13 133 "shipped_days": 13
134 } 134 }
135 ], 135 ],
136− "expected": 9108.6 136+ "expected": 9456.8
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -237,7 +237,7 @@
237 "shipped_days": 6 237 "shipped_days": 6
238 } 238 }
239 ], 239 ],
240− "expected": 9400.8 240+ "expected": 9760.9
241 }, 241 },
242 { 242 {
243 "input": [ 243 "input": [
@@ -413,7 +413,7 @@
413 "shipped_days": 1 413 "shipped_days": 1
414 } 414 }
415 ], 415 ],
416− "expected": 15622.3 416+ "expected": 16236.8
417 }, 417 },
418 { 418 {
419 "input": [ 419 "input": [
@@ -589,7 +589,7 @@
589 "shipped_days": 1 589 "shipped_days": 1
590 } 590 }
591 ], 591 ],
592− "expected": 14227.3 592+ "expected": 14862.0
593 }, 593 },
594 { 594 {
595 "input": [ 595 "input": [
@@ -756,7 +756,7 @@
756 "shipped_days": 8 756 "shipped_days": 8
757 } 757 }
758 ], 758 ],
759− "expected": 15622.3 759+ "expected": 16236.8
760 } 760 }
761 ] 761 ]
762 } 762 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00018_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00018_2000", 2 "document_id": "exec_00018_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "6f0951d7af338593059efbe431f0b7ce5bf9ae652e3dfb9eedafa7e192b80c01", 5+ "document_hash": "f219110fda4e5d20fabbe592248da3e78f61f411dbc16b9412d9b8d692eda870",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 "shipped_days": 5 160 "shipped_days": 5
161 } 161 }
162 ], 162 ],
163− "expected": 9488.91 163+ "expected": 9778.64
164 }, 164 },
165 { 165 {
166 "input": [ 166 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 10 228 "shipped_days": 10
229 } 229 }
230 ], 230 ],
231− "expected": 4726.7 231+ "expected": 4840.02
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -395,7 +395,7 @@
395 "shipped_days": 7 395 "shipped_days": 7
396 } 396 }
397 ], 397 ],
398− "expected": 12483.47 398+ "expected": 12743.47
399 }, 399 },
400 { 400 {
401 "input": [ 401 "input": [
@@ -580,7 +580,7 @@
580 "shipped_days": 12 580 "shipped_days": 12
581 } 581 }
582 ], 582 ],
583− "expected": 14020.45 583+ "expected": 14355.94
584 }, 584 },
585 { 585 {
586 "input": [ 586 "input": [
@@ -648,7 +648,7 @@
648 "shipped_days": 4 648 "shipped_days": 4
649 } 649 }
650 ], 650 ],
651− "expected": 5979.52 651+ "expected": 6238.62
652 } 652 }
653 ] 653 ]
654 } 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00019_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00019_2000", 2 "document_id": "exec_00019_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "733853d52a559dfc67d0cdab02de9e21c2aa09e0b25702d2cadfb38c20ab2159", 5+ "document_hash": "232819b55ebc60215845a14aaf01c04a94dc7902933c03a47546ec41ee16502f",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -151,7 +151,7 @@
151 "shipped_days": 5 151 "shipped_days": 5
152 } 152 }
153 ], 153 ],
154− "expected": 6814.82 154+ "expected": 7072.24
155 }, 155 },
156 { 156 {
157 "input": [ 157 "input": [
@@ -318,7 +318,7 @@
318 "shipped_days": 12 318 "shipped_days": 12
319 } 319 }
320 ], 320 ],
321− "expected": 11999.36 321+ "expected": 12263.98
322 }, 322 },
323 { 323 {
324 "input": [ 324 "input": [
@@ -404,7 +404,7 @@
404 "shipped_days": 6 404 "shipped_days": 6
405 } 405 }
406 ], 406 ],
407− "expected": 4050.38 407+ "expected": 4142.87
408 }, 408 },
409 { 409 {
410 "input": [ 410 "input": [
@@ -544,7 +544,7 @@
544 "shipped_days": 1 544 "shipped_days": 1
545 } 545 }
546 ], 546 ],
547− "expected": 12701.97 547+ "expected": 13019.21
548 }, 548 },
549 { 549 {
550 "input": [ 550 "input": [
@@ -630,7 +630,7 @@
630 "shipped_days": 7 630 "shipped_days": 7
631 } 631 }
632 ], 632 ],
633− "expected": 5014.11 633+ "expected": 5141.41
634 } 634 }
635 ] 635 ]
636 } 636 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00020_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00020_2000", 2 "document_id": "exec_00020_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "train", 4 "split": "train",
5− "document_hash": "1a7bfffc3347254640c8b76b9b5f0ff368a758543995fa06006e28ea43d8eb71", 5+ "document_hash": "862136f6ccb7f77097086ceddd84945e55b7df17401a63b0ce306740969966e5",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -223,7 +223,7 @@
223 "shipped_days": 5 223 "shipped_days": 5
224 } 224 }
225 ], 225 ],
226− "expected": 8844.77 226+ "expected": 9290.06
227 }, 227 },
228 { 228 {
229 "input": [ 229 "input": [
@@ -435,7 +435,7 @@
435 "shipped_days": 13 435 "shipped_days": 13
436 } 436 }
437 ], 437 ],
438− "expected": 9166.7 438+ "expected": 9612.0
439 }, 439 },
440 { 440 {
441 "input": [ 441 "input": [
@@ -530,7 +530,7 @@
530 "shipped_days": 4 530 "shipped_days": 4
531 } 531 }
532 ], 532 ],
533− "expected": 3537.06 533+ "expected": 3697.66
534 }, 534 },
535 { 535 {
536 "input": [ 536 "input": [
@@ -715,7 +715,7 @@
715 "shipped_days": 13 715 "shipped_days": 13
716 } 716 }
717 ], 717 ],
718− "expected": 9068.02 718+ "expected": 9513.31
719 }, 719 },
720 { 720 {
721 "input": [ 721 "input": [
@@ -891,7 +891,7 @@
891 "shipped_days": 9 891 "shipped_days": 9
892 } 892 }
893 ], 893 ],
894− "expected": 8860.27 894+ "expected": 9289.44
895 } 895 }
896 ] 896 ]
897 } 897 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00101_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00101_2000", 2 "document_id": "exec_00101_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", 5+ "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 2, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 3, \"min_units\": 2, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -223,7 +223,7 @@
223 "shipped_days": 1 223 "shipped_days": 1
224 } 224 }
225 ], 225 ],
226− "expected": 10226.1 226+ "expected": 10668.21
227 }, 227 },
228 { 228 {
229 "input": [ 229 "input": [
@@ -381,7 +381,7 @@
381 "shipped_days": 6 381 "shipped_days": 6
382 } 382 }
383 ], 383 ],
384− "expected": 8617.57 384+ "expected": 9061.05
385 }, 385 },
386 { 386 {
387 "input": [ 387 "input": [
@@ -458,7 +458,7 @@
458 "shipped_days": 2 458 "shipped_days": 2
459 } 459 }
460 ], 460 ],
461− "expected": 5081.68 461+ "expected": 5250.95
462 }, 462 },
463 { 463 {
464 "input": [ 464 "input": [
@@ -526,7 +526,7 @@
526 "shipped_days": 1 526 "shipped_days": 1
527 } 527 }
528 ], 528 ],
529− "expected": 3365.1 529+ "expected": 3549.55
530 }, 530 },
531 { 531 {
532 "input": [ 532 "input": [
@@ -684,7 +684,7 @@
684 "shipped_days": 8 684 "shipped_days": 8
685 } 685 }
686 ], 686 ],
687− "expected": 5902.78 687+ "expected": 6132.87
688 } 688 }
689 ] 689 ]
690 } 690 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00102_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00102_2000", 2 "document_id": "exec_00102_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "5535ac3109616f7d46dff5417535ca232754aee2c0759e072d552e465f004d46", 5+ "document_hash": "61b6c7bb86261a26bf000996edd5f9326f7f77ca1c37085fd999cddf1bff2396",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -70,7 +70,7 @@
70 "shipped_days": 12 70 "shipped_days": 12
71 } 71 }
72 ], 72 ],
73− "expected": 2436.53 73+ "expected": 2584.83
74 }, 74 },
75 { 75 {
76 "input": [ 76 "input": [
@@ -192,7 +192,7 @@
192 "shipped_days": 7 192 "shipped_days": 7
193 } 193 }
194 ], 194 ],
195− "expected": 7499.55 195+ "expected": 7698.08
196 }, 196 },
197 { 197 {
198 "input": [ 198 "input": [
@@ -386,7 +386,7 @@
386 "shipped_days": 12 386 "shipped_days": 12
387 } 387 }
388 ], 388 ],
389− "expected": 9927.27 389+ "expected": 10303.84
390 }, 390 },
391 { 391 {
392 "input": [ 392 "input": [
@@ -436,7 +436,7 @@
436 "shipped_days": 9 436 "shipped_days": 9
437 } 437 }
438 ], 438 ],
439− "expected": 547.82 439+ "expected": 579.97
440 }, 440 },
441 { 441 {
442 "input": [ 442 "input": [
@@ -495,7 +495,7 @@
495 "shipped_days": 3 495 "shipped_days": 3
496 } 496 }
497 ], 497 ],
498− "expected": 3172.53 498+ "expected": 3203.35
499 } 499 }
500 ] 500 ]
501 } 501 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00103_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00103_2000", 2 "document_id": "exec_00103_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "a7875d1e95742c6246c4ec3862b9797a4269a2b607e47869f5014d7dc29139e0", 5+ "document_hash": "596ef1cd6d37fdac729520a738ed129e4ffaf619d372ad869d8188a0a5cac1ef",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 2, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 8, \"min_units\": 2, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 10 133 "shipped_days": 10
134 } 134 }
135 ], 135 ],
136− "expected": 6268.0 136+ "expected": 6449.0
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -282,7 +282,7 @@
282 "shipped_days": 8 282 "shipped_days": 8
283 } 283 }
284 ], 284 ],
285− "expected": 8227.0 285+ "expected": 8477.0
286 }, 286 },
287 { 287 {
288 "input": [ 288 "input": [
@@ -350,7 +350,7 @@
350 "shipped_days": 8 350 "shipped_days": 8
351 } 351 }
352 ], 352 ],
353− "expected": 5279.0 353+ "expected": 5424.0
354 }, 354 },
355 { 355 {
356 "input": [ 356 "input": [
@@ -459,7 +459,7 @@
459 "shipped_days": 4 459 "shipped_days": 4
460 } 460 }
461 ], 461 ],
462− "expected": 5088.0 462+ "expected": 5269.0
463 } 463 }
464 ] 464 ]
465 } 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00104_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00104_2000", 2 "document_id": "exec_00104_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "bce3091895fc315bf981ac1272a7da1f00b0f64819f1ecfa2a6dd8a3d23f2207", 5+ "document_hash": "6b4aadaf1b71672d6d8aed5a412c7aa490e3e0eb1c92e5df6db1cc9554f83483",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 9 88 "shipped_days": 9
89 } 89 }
90 ], 90 ],
91− "expected": 2111.0 91+ "expected": 2225.0
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -138,7 +138,7 @@
138 "shipped_days": 8 138 "shipped_days": 8
139 } 139 }
140 ], 140 ],
141− "expected": 2304.0 141+ "expected": 2439.0
142 }, 142 },
143 { 143 {
144 "input": [ 144 "input": [
@@ -215,7 +215,7 @@
215 "shipped_days": 10 215 "shipped_days": 10
216 } 216 }
217 ], 217 ],
218− "expected": 6214.0 218+ "expected": 6444.0
219 }, 219 },
220 { 220 {
221 "input": [ 221 "input": [
@@ -373,7 +373,7 @@
373 "shipped_days": 12 373 "shipped_days": 12
374 } 374 }
375 ], 375 ],
376− "expected": 8777.0 376+ "expected": 9137.0
377 }, 377 },
378 { 378 {
379 "input": [ 379 "input": [
@@ -441,7 +441,7 @@
441 "shipped_days": 3 441 "shipped_days": 3
442 } 442 }
443 ], 443 ],
444− "expected": 3400.0 444+ "expected": 3456.0
445 } 445 }
446 ] 446 ]
447 } 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00105_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00105_2000", 2 "document_id": "exec_00105_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "fe4b8857bf1774be1123bec85a5743da5c3551d53125ddf128efde401f3990c2", 5+ "document_hash": "a59face90fe592d3d8a9b375aafefb775631dd798a60a424f981cceae1ec2996",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 1 133 "shipped_days": 1
134 } 134 }
135 ], 135 ],
136− "expected": 10771.09 136+ "expected": 10902.26
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -237,7 +237,7 @@
237 "shipped_days": 3 237 "shipped_days": 3
238 } 238 }
239 ], 239 ],
240− "expected": 10110.37 240+ "expected": 10248.58
241 }, 241 },
242 { 242 {
243 "input": [ 243 "input": [
@@ -359,7 +359,7 @@
359 "shipped_days": 11 359 "shipped_days": 11
360 } 360 }
361 ], 361 ],
362− "expected": 11426.01 362+ "expected": 11578.59
363 }, 363 },
364 { 364 {
365 "input": [ 365 "input": [
@@ -445,7 +445,7 @@
445 "shipped_days": 10 445 "shipped_days": 10
446 } 446 }
447 ], 447 ],
448− "expected": 7450.92 448+ "expected": 7589.13
449 }, 449 },
450 { 450 {
451 "input": [ 451 "input": [
@@ -513,7 +513,7 @@
513 "shipped_days": 8 513 "shipped_days": 8
514 } 514 }
515 ], 515 ],
516− "expected": 7375.01 516+ "expected": 7509.36
517 } 517 }
518 ] 518 ]
519 } 519 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00106_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00106_2000", 2 "document_id": "exec_00106_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "c3652bfa1632f0cc62d6512131ef519100143881225c8b4ee572c1876e7c6c7f", 5+ "document_hash": "a78f55af374fd04eb589b193a9eca2f84cf04bcdb3fa60aaeb29fbe358c79464",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 7, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 758 30+ "char_end": 631
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 "shipped_days": 8 79 "shipped_days": 8
80 } 80 }
81 ], 81 ],
82− "expected": 5525.0 82+ "expected": 5732.0
83 }, 83 },
84 { 84 {
85 "input": [ 85 "input": [
@@ -174,7 +174,7 @@
174 "shipped_days": 2 174 "shipped_days": 2
175 } 175 }
176 ], 176 ],
177− "expected": 6501.0 177+ "expected": 6608.0
178 }, 178 },
179 { 179 {
180 "input": [ 180 "input": [
@@ -287,7 +287,7 @@
287 "shipped_days": 2 287 "shipped_days": 2
288 } 288 }
289 ], 289 ],
290− "expected": 10681.0 290+ "expected": 11051.0
291 }, 291 },
292 { 292 {
293 "input": [ 293 "input": [
@@ -337,7 +337,7 @@
337 "shipped_days": 1 337 "shipped_days": 1
338 } 338 }
339 ], 339 ],
340− "expected": 3956.0 340+ "expected": 4119.0
341 }, 341 },
342 { 342 {
343 "input": [ 343 "input": [
@@ -468,7 +468,7 @@
468 "shipped_days": 14 468 "shipped_days": 14
469 } 469 }
470 ], 470 ],
471− "expected": 11106.0 471+ "expected": 11475.0
472 } 472 }
473 ] 473 ]
474 } 474 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00107_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00107_2000", 2 "document_id": "exec_00107_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "983dbcacdfae2acf0316690a7fe50cfa2523ae41beffd9de42356f2ebd3d1e79", 5+ "document_hash": "83ea6e4f6c5d3a6eed7a921a930819d760ffb3dac7c22761dc9b4c28084904cc",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 "shipped_days": 14 106 "shipped_days": 14
107 } 107 }
108 ], 108 ],
109− "expected": 5613.04 109+ "expected": 5823.85
110 }, 110 },
111 { 111 {
112 "input": [ 112 "input": [
@@ -210,7 +210,7 @@
210 "shipped_days": 3 210 "shipped_days": 3
211 } 211 }
212 ], 212 ],
213− "expected": 3646.24 213+ "expected": 3759.3
214 }, 214 },
215 { 215 {
216 "input": [ 216 "input": [
@@ -260,7 +260,7 @@
260 "shipped_days": 13 260 "shipped_days": 13
261 } 261 }
262 ], 262 ],
263− "expected": 1237.55 263+ "expected": 1292.08
264 }, 264 },
265 { 265 {
266 "input": [ 266 "input": [
@@ -373,7 +373,7 @@
373 "shipped_days": 3 373 "shipped_days": 3
374 } 374 }
375 ], 375 ],
376− "expected": 6615.56 376+ "expected": 6884.9
377 }, 377 },
378 { 378 {
379 "input": [ 379 "input": [
@@ -459,7 +459,7 @@
459 "shipped_days": 4 459 "shipped_days": 4
460 } 460 }
461 ], 461 ],
462− "expected": 3551.44 462+ "expected": 3657.72
463 } 463 }
464 ] 464 ]
465 } 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00108_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00108_2000", 2 "document_id": "exec_00108_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "7ebb05d3714e237d074dfc9a69c5b83e99932f9041b5d4fe15c8fc7224ba7078", 5+ "document_hash": "b9ccaaf72598ea6e12a10942f2697dd01ca891115b652043a2f6ce4cb68570cc",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 "shipped_days": 8 169 "shipped_days": 8
170 } 170 }
171 ], 171 ],
172− "expected": 4576.0 172+ "expected": 4658.0
173 }, 173 },
174 { 174 {
175 "input": [ 175 "input": [
@@ -345,7 +345,7 @@
345 "shipped_days": 1 345 "shipped_days": 1
346 } 346 }
347 ], 347 ],
348− "expected": 8240.0 348+ "expected": 8534.0
349 }, 349 },
350 { 350 {
351 "input": [ 351 "input": [
@@ -476,7 +476,7 @@
476 "shipped_days": 6 476 "shipped_days": 6
477 } 477 }
478 ], 478 ],
479− "expected": 8190.0 479+ "expected": 8483.0
480 }, 480 },
481 { 481 {
482 "input": [ 482 "input": [
@@ -553,7 +553,7 @@
553 "shipped_days": 13 553 "shipped_days": 13
554 } 554 }
555 ], 555 ],
556− "expected": 4773.0 556+ "expected": 4985.0
557 }, 557 },
558 { 558 {
559 "input": [ 559 "input": [
@@ -702,7 +702,7 @@
702 "shipped_days": 1 702 "shipped_days": 1
703 } 703 }
704 ], 704 ],
705− "expected": 8587.0 705+ "expected": 8880.0
706 } 706 }
707 ] 707 ]
708 } 708 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00109_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00109_2000", 2 "document_id": "exec_00109_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "680dab47e67683cb351941eea647197cb2b2ea472ff116582019ace0c543d461", 5+ "document_hash": "d53ada6a135b4827dcacb2ebec142b5000ad21961fb0cd199fc7c609424d4d5a",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 "shipped_days": 14 79 "shipped_days": 14
80 } 80 }
81 ], 81 ],
82− "expected": 3510.0 82+ "expected": 3706.0
83 }, 83 },
84 { 84 {
85 "input": [ 85 "input": [
@@ -255,7 +255,7 @@
255 "shipped_days": 4 255 "shipped_days": 4
256 } 256 }
257 ], 257 ],
258− "expected": 11820.0 258+ "expected": 12136.0
259 }, 259 },
260 { 260 {
261 "input": [ 261 "input": [
@@ -296,7 +296,7 @@
296 "shipped_days": 11 296 "shipped_days": 11
297 } 297 }
298 ], 298 ],
299− "expected": 1937.0 299+ "expected": 2020.0
300 }, 300 },
301 { 301 {
302 "input": [ 302 "input": [
@@ -490,7 +490,7 @@
490 "shipped_days": 3 490 "shipped_days": 3
491 } 491 }
492 ], 492 ],
493− "expected": 12167.0 493+ "expected": 12483.0
494 }, 494 },
495 { 495 {
496 "input": [ 496 "input": [
@@ -585,7 +585,7 @@
585 "shipped_days": 14 585 "shipped_days": 14
586 } 586 }
587 ], 587 ],
588− "expected": 6741.0 588+ "expected": 6937.0
589 } 589 }
590 ] 590 ]
591 } 591 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00110_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00110_2000", 2 "document_id": "exec_00110_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "f1699da28903736ad58ec0640a277bb2eac030c80c49c22c027c273374df86fa", 5+ "document_hash": "789f1f1a750d514137a988d21a0807cc30c8548da830f2eeef66854fc179fcf5",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 "shipped_days": 13 115 "shipped_days": 13
116 } 116 }
117 ], 117 ],
118− "expected": 2826.7 118+ "expected": 2987.2
119 }, 119 },
120 { 120 {
121 "input": [ 121 "input": [
@@ -237,7 +237,7 @@
237 "shipped_days": 2 237 "shipped_days": 2
238 } 238 }
239 ], 239 ],
240− "expected": 7282.5 240+ "expected": 7731.2
241 }, 241 },
242 { 242 {
243 "input": [ 243 "input": [
@@ -296,7 +296,7 @@
296 "shipped_days": 5 296 "shipped_days": 5
297 } 297 }
298 ], 298 ],
299− "expected": 2365.4 299+ "expected": 2478.7
300 }, 300 },
301 { 301 {
302 "input": [ 302 "input": [
@@ -391,7 +391,7 @@
391 "shipped_days": 7 391 "shipped_days": 7
392 } 392 }
393 ], 393 ],
394− "expected": 3707.7 394+ "expected": 3903.5
395 }, 395 },
396 { 396 {
397 "input": [ 397 "input": [
@@ -450,7 +450,7 @@
450 "shipped_days": 7 450 "shipped_days": 7
451 } 451 }
452 ], 452 ],
453− "expected": 1578.9 453+ "expected": 1670.6
454 } 454 }
455 ] 455 ]
456 } 456 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00111_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00111_2000", 2 "document_id": "exec_00111_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "e65e9534b60b15cdaadf465396c6385868f5a0ed4be5ba04c458937c2eff43ae", 5+ "document_hash": "9f3d9d2caa1830b0bf0b983179f4de3251ec9bb8648c710bf43f0fb6c531ef85",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 750 30+ "char_end": 623
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 "shipped_days": 14 79 "shipped_days": 14
80 } 80 }
81 ], 81 ],
82− "expected": 4912.9 82+ "expected": 4941.8
83 }, 83 },
84 { 84 {
85 "input": [ 85 "input": [
@@ -129,7 +129,7 @@
129 "shipped_days": 4 129 "shipped_days": 4
130 } 130 }
131 ], 131 ],
132− "expected": 4597.7 132+ "expected": 4704.8
133 }, 133 },
134 { 134 {
135 "input": [ 135 "input": [
@@ -242,7 +242,7 @@
242 "shipped_days": 2 242 "shipped_days": 2
243 } 243 }
244 ], 244 ],
245− "expected": 7960.9 245+ "expected": 8086.8
246 }, 246 },
247 { 247 {
248 "input": [ 248 "input": [
@@ -283,7 +283,7 @@
283 "shipped_days": 2 283 "shipped_days": 2
284 } 284 }
285 ], 285 ],
286− "expected": 4797.6 286+ "expected": 4924.0
287 }, 287 },
288 { 288 {
289 "input": [ 289 "input": [
@@ -333,7 +333,7 @@
333 "shipped_days": 1 333 "shipped_days": 1
334 } 334 }
335 ], 335 ],
336− "expected": 2120.4 336+ "expected": 2221.0
337 } 337 }
338 ] 338 ]
339 } 339 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00112_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00112_2000", 2 "document_id": "exec_00112_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "4a8db801f5e7f6b2108618514f90c37fbdbb61e902c4441c0d68548a5792da05", 5+ "document_hash": "182cf1cbb5939b7e1ca9170096aa77a9cc3c2c4f8efb3195e8b4ad19b77a3f46",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 5, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 5, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 759 30+ "char_end": 632
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 2 88 "shipped_days": 2
89 } 89 }
90 ], 90 ],
91− "expected": 3649.3 91+ "expected": 3759.0
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -165,7 +165,7 @@
165 "shipped_days": 2 165 "shipped_days": 2
166 } 166 }
167 ], 167 ],
168− "expected": 7068.7 168+ "expected": 7383.5
169 }, 169 },
170 { 170 {
171 "input": [ 171 "input": [
@@ -206,7 +206,7 @@
206 "shipped_days": 6 206 "shipped_days": 6
207 } 207 }
208 ], 208 ],
209− "expected": 1034.9 209+ "expected": 1091.9
210 }, 210 },
211 { 211 {
212 "input": [ 212 "input": [
@@ -310,7 +310,7 @@
310 "shipped_days": 3 310 "shipped_days": 3
311 } 311 }
312 ], 312 ],
313− "expected": 6354.2 313+ "expected": 6637.2
314 }, 314 },
315 { 315 {
316 "input": [ 316 "input": [
@@ -432,7 +432,7 @@
432 "shipped_days": 2 432 "shipped_days": 2
433 } 433 }
434 ], 434 ],
435− "expected": 8695.4 435+ "expected": 9104.1
436 } 436 }
437 ] 437 ]
438 } 438 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00113_2000.json +6 −6
@@ -2,13 +2,13 @@
2 "document_id": "exec_00113_2000", 2 "document_id": "exec_00113_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "7535b4d324a3e28b457e40fe41d18d52e75a9ed0271b411285e522c6c8c8200e", 5+ "document_hash": "c89720bd112bca637c8fa3b4a98f7cc41a384be468eee47f5c44607ee6fc46f1",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -368,7 +368,7 @@
368 "shipped_days": 12 368 "shipped_days": 12
369 } 369 }
370 ], 370 ],
371− "expected": 7773.0 371+ "expected": 7875.8
372 }, 372 },
373 { 373 {
374 "input": [ 374 "input": [
@@ -544,7 +544,7 @@
544 "shipped_days": 6 544 "shipped_days": 6
545 } 545 }
546 ], 546 ],
547− "expected": 9327.9 547+ "expected": 9430.8
548 }, 548 },
549 { 549 {
550 "input": [ 550 "input": [
modified data/generated/keys/exec/exec_00114_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00114_2000", 2 "document_id": "exec_00114_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "e1faab8175075b8cf98d2e5e16181db3f5a8f103ed89d35efee7ba4684968699", 5+ "document_hash": "86a377fe8f0559479ca21128972c50d14bd9e19d38d2bf49f763d402259c0aad",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 7 124 "shipped_days": 7
125 } 125 }
126 ], 126 ],
127− "expected": 3625.0 127+ "expected": 3805.0
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -183,7 +183,7 @@
183 "shipped_days": 6 183 "shipped_days": 6
184 } 184 }
185 ], 185 ],
186− "expected": 2378.0 186+ "expected": 2511.0
187 }, 187 },
188 { 188 {
189 "input": [ 189 "input": [
@@ -224,7 +224,7 @@
224 "shipped_days": 1 224 "shipped_days": 1
225 } 225 }
226 ], 226 ],
227− "expected": 2182.0 227+ "expected": 2273.0
228 }, 228 },
229 { 229 {
230 "input": [ 230 "input": [
@@ -265,7 +265,7 @@
265 "shipped_days": 7 265 "shipped_days": 7
266 } 266 }
267 ], 267 ],
268− "expected": 1604.0 268+ "expected": 1697.0
269 }, 269 },
270 { 270 {
271 "input": [ 271 "input": [
@@ -324,7 +324,7 @@
324 "shipped_days": 1 324 "shipped_days": 1
325 } 325 }
326 ], 326 ],
327− "expected": 1570.0 327+ "expected": 1635.0
328 } 328 }
329 ] 329 ]
330 } 330 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00115_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00115_2000", 2 "document_id": "exec_00115_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "4d6a9aa137c992493ef42f6758dbb35b873a4a392b00eae7152d6e45b68a3c1b", 5+ "document_hash": "33ee1d20ea5713048e1335d3786ba0b10b3a5998a16200f09823d0c4cf0dcbe1",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 2, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 2, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 "shipped_days": 1 97 "shipped_days": 1
98 } 98 }
99 ], 99 ],
100− "expected": 6895.0 100+ "expected": 7259.0
101 }, 101 },
102 { 102 {
103 "input": [ 103 "input": [
@@ -237,7 +237,7 @@
237 "shipped_days": 10 237 "shipped_days": 10
238 } 238 }
239 ], 239 ],
240− "expected": 8291.0 240+ "expected": 8731.0
241 }, 241 },
242 { 242 {
243 "input": [ 243 "input": [
@@ -305,7 +305,7 @@
305 "shipped_days": 13 305 "shipped_days": 13
306 } 306 }
307 ], 307 ],
308− "expected": 1592.0 308+ "expected": 1662.0
309 }, 309 },
310 { 310 {
311 "input": [ 311 "input": [
@@ -364,7 +364,7 @@
364 "shipped_days": 1 364 "shipped_days": 1
365 } 365 }
366 ], 366 ],
367− "expected": 1981.0 367+ "expected": 2073.0
368 }, 368 },
369 { 369 {
370 "input": [ 370 "input": [
@@ -468,7 +468,7 @@
468 "shipped_days": 4 468 "shipped_days": 4
469 } 469 }
470 ], 470 ],
471− "expected": 2518.0 471+ "expected": 2593.0
472 } 472 }
473 ] 473 ]
474 } 474 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00116_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00116_2000", 2 "document_id": "exec_00116_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "2b6100ff4b04865ebf622b9749757f3b32215faa2e32f12d63be71b0c18fc7be", 5+ "document_hash": "379dcf9e9ada65463c8d73e2cbb54ff33c8c46ea3154539a1d6df25bab2a0a0e",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 5, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 5, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 11 124 "shipped_days": 11
125 } 125 }
126 ], 126 ],
127− "expected": 3800.17 127+ "expected": 4017.88
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -219,7 +219,7 @@
219 "shipped_days": 11 219 "shipped_days": 11
220 } 220 }
221 ], 221 ],
222− "expected": 3800.17 222+ "expected": 4017.88
223 }, 223 },
224 { 224 {
225 "input": [ 225 "input": [
@@ -305,7 +305,7 @@
305 "shipped_days": 4 305 "shipped_days": 4
306 } 306 }
307 ], 307 ],
308− "expected": 2139.86 308+ "expected": 2256.71
309 }, 309 },
310 { 310 {
311 "input": [ 311 "input": [
@@ -364,7 +364,7 @@
364 "shipped_days": 4 364 "shipped_days": 4
365 } 365 }
366 ], 366 ],
367− "expected": 2238.1 367+ "expected": 2365.02
368 }, 368 },
369 { 369 {
370 "input": [ 370 "input": [
@@ -477,7 +477,7 @@
477 "shipped_days": 8 477 "shipped_days": 8
478 } 478 }
479 ], 479 ],
480− "expected": 3800.17 480+ "expected": 4017.88
481 } 481 }
482 ] 482 ]
483 } 483 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00117_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00117_2000", 2 "document_id": "exec_00117_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "6e53d886a35d8e51c98908f1e40e9314353c62a19e3197603509fd2a59c6bbf3", 5+ "document_hash": "34a1cc9614d0c2d2f9766b2968ae3fa3eb2b6cc1a372750650ae7f7359325896",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 "shipped_days": 10 169 "shipped_days": 10
170 } 170 }
171 ], 171 ],
172− "expected": 5636.0 172+ "expected": 5781.7
173 }, 173 },
174 { 174 {
175 "input": [ 175 "input": [
@@ -318,7 +318,7 @@
318 "shipped_days": 2 318 "shipped_days": 2
319 } 319 }
320 ], 320 ],
321− "expected": 5209.9 321+ "expected": 5379.2
322 }, 322 },
323 { 323 {
324 "input": [ 324 "input": [
@@ -386,7 +386,7 @@
386 "shipped_days": 2 386 "shipped_days": 2
387 } 387 }
388 ], 388 ],
389− "expected": 468.5 389+ "expected": 493.1
390 }, 390 },
391 { 391 {
392 "input": [ 392 "input": [
@@ -553,7 +553,7 @@
553 "shipped_days": 3 553 "shipped_days": 3
554 } 554 }
555 ], 555 ],
556− "expected": 6038.0 556+ "expected": 6207.3
557 }, 557 },
558 { 558 {
559 "input": [ 559 "input": [
modified data/generated/keys/exec/exec_00118_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00118_2000", 2 "document_id": "exec_00118_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "5353584056feb451c43bd2199aa87e2f7064bbfb9485b6eb1f6a1387dddac0f6", 5+ "document_hash": "58d2108f993d9baafb67222dc4bd97895639cb6c2410e675e3ed2c4233c8fee4",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -255,7 +255,7 @@
255 "shipped_days": 9 255 "shipped_days": 9
256 } 256 }
257 ], 257 ],
258− "expected": 10422.52 258+ "expected": 10515.77
259 }, 259 },
260 { 260 {
261 "input": [ 261 "input": [
@@ -377,7 +377,7 @@
377 "shipped_days": 5 377 "shipped_days": 5
378 } 378 }
379 ], 379 ],
380− "expected": 9268.75 380+ "expected": 9362.0
381 }, 381 },
382 { 382 {
383 "input": [ 383 "input": [
@@ -454,7 +454,7 @@
454 "shipped_days": 1 454 "shipped_days": 1
455 } 455 }
456 ], 456 ],
457− "expected": 7023.17 457+ "expected": 7059.96
458 }, 458 },
459 { 459 {
460 "input": [ 460 "input": [
@@ -594,7 +594,7 @@
594 "shipped_days": 1 594 "shipped_days": 1
595 } 595 }
596 ], 596 ],
597− "expected": 10422.52 597+ "expected": 10515.77
598 } 598 }
599 ] 599 ]
600 } 600 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00119_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00119_2000", 2 "document_id": "exec_00119_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "f62c9b5ecd91caa0cf95af49bf6d40dd2bc3885255008434f482e56b7a2a2fae", 5+ "document_hash": "98f121e7a33ff30acfcaf7a0346bbc04cc9f508e092e316490793765a8e0ab09",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 3 133 "shipped_days": 3
134 } 134 }
135 ], 135 ],
136− "expected": 3946.0 136+ "expected": 4007.0
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -282,7 +282,7 @@
282 "shipped_days": 14 282 "shipped_days": 14
283 } 283 }
284 ], 284 ],
285− "expected": 7910.0 285+ "expected": 8022.0
286 }, 286 },
287 { 287 {
288 "input": [ 288 "input": [
@@ -386,7 +386,7 @@
386 "shipped_days": 8 386 "shipped_days": 8
387 } 387 }
388 ], 388 ],
389− "expected": 4889.0 389+ "expected": 4951.0
390 }, 390 },
391 { 391 {
392 "input": [ 392 "input": [
@@ -499,7 +499,7 @@
499 "shipped_days": 8 499 "shipped_days": 8
500 } 500 }
501 ], 501 ],
502− "expected": 4861.0 502+ "expected": 4973.0
503 }, 503 },
504 { 504 {
505 "input": [ 505 "input": [
@@ -630,7 +630,7 @@
630 "shipped_days": 5 630 "shipped_days": 5
631 } 631 }
632 ], 632 ],
633− "expected": 7046.0 633+ "expected": 7147.0
634 } 634 }
635 ] 635 ]
636 } 636 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00120_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00120_2000", 2 "document_id": "exec_00120_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "5b1b1089a793b0953579b8ce004efe99282ffd8a6eb17b16c3ce50273c121aad", 5+ "document_hash": "c5be33e5009123ba2c251884304b839ae6fc5163f0745c302cffbabf618b40cb",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 5, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 5, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 "shipped_days": 6 169 "shipped_days": 6
170 } 170 }
171 ], 171 ],
172− "expected": 11463.0 172+ "expected": 11919.2
173 }, 173 },
174 { 174 {
175 "input": [ 175 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 8 228 "shipped_days": 8
229 } 229 }
230 ], 230 ],
231− "expected": 3698.4 231+ "expected": 3926.1
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -323,7 +323,7 @@
323 "shipped_days": 1 323 "shipped_days": 1
324 } 324 }
325 ], 325 ],
326− "expected": 7350.4 326+ "expected": 7625.0
327 }, 327 },
328 { 328 {
329 "input": [ 329 "input": [
@@ -499,7 +499,7 @@
499 "shipped_days": 10 499 "shipped_days": 10
500 } 500 }
501 ], 501 ],
502− "expected": 11559.0 502+ "expected": 12015.2
503 }, 503 },
504 { 504 {
505 "input": [ 505 "input": [
@@ -666,7 +666,7 @@
666 "shipped_days": 10 666 "shipped_days": 10
667 } 667 }
668 ], 668 ],
669− "expected": 8903.5 669+ "expected": 9203.5
670 } 670 }
671 ] 671 ]
672 } 672 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00121_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00121_2000", 2 "document_id": "exec_00121_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "9064e893e331010c27db6e1229c2af7da83ed7fb3680e079fc5c1a9ee7b2d73a", 5+ "document_hash": "1645df3901b5bd61ea8c6790e15a695b1144eef69bc075121ffd0c83e6fc13af",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 "shipped_days": 3 106 "shipped_days": 3
107 } 107 }
108 ], 108 ],
109− "expected": 2869.2 109+ "expected": 2888.6
110 }, 110 },
111 { 111 {
112 "input": [ 112 "input": [
@@ -147,7 +147,7 @@
147 "shipped_days": 12 147 "shipped_days": 12
148 } 148 }
149 ], 149 ],
150− "expected": 3908.9 150+ "expected": 4089.0
151 }, 151 },
152 { 152 {
153 "input": [ 153 "input": [
@@ -287,7 +287,7 @@
287 "shipped_days": 5 287 "shipped_days": 5
288 } 288 }
289 ], 289 ],
290− "expected": 6986.1 290+ "expected": 7212.7
291 }, 291 },
292 { 292 {
293 "input": [ 293 "input": [
@@ -409,7 +409,7 @@
409 "shipped_days": 10 409 "shipped_days": 10
410 } 410 }
411 ], 411 ],
412− "expected": 8197.2 412+ "expected": 8385.9
413 }, 413 },
414 { 414 {
415 "input": [ 415 "input": [
@@ -504,7 +504,7 @@
504 "shipped_days": 4 504 "shipped_days": 4
505 } 505 }
506 ], 506 ],
507− "expected": 6209.1 507+ "expected": 6372.1
508 } 508 }
509 ] 509 ]
510 } 510 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00122_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00122_2000", 2 "document_id": "exec_00122_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "8641f3a09b033f2a216948ea197682c280cda7a9559b7102473da5f152a73eb3", 5+ "document_hash": "bd26e436a86876d04961cc142588671db4ead753f838933bde6907da5354f70e",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -142,7 +142,7 @@
142 "shipped_days": 8 142 "shipped_days": 8
143 } 143 }
144 ], 144 ],
145− "expected": 6993.4 145+ "expected": 7221.2
146 }, 146 },
147 { 147 {
148 "input": [ 148 "input": [
@@ -345,7 +345,7 @@
345 "shipped_days": 7 345 "shipped_days": 7
346 } 346 }
347 ], 347 ],
348− "expected": 12417.7 348+ "expected": 12645.5
349 }, 349 },
350 { 350 {
351 "input": [ 351 "input": [
@@ -440,7 +440,7 @@
440 "shipped_days": 4 440 "shipped_days": 4
441 } 441 }
442 ], 442 ],
443− "expected": 4310.8 443+ "expected": 4400.2
444 }, 444 },
445 { 445 {
446 "input": [ 446 "input": [
@@ -535,7 +535,7 @@
535 "shipped_days": 8 535 "shipped_days": 8
536 } 536 }
537 ], 537 ],
538− "expected": 7689.7 538+ "expected": 7967.8
539 }, 539 },
540 { 540 {
541 "input": [ 541 "input": [
@@ -576,7 +576,7 @@
576 "shipped_days": 12 576 "shipped_days": 12
577 } 577 }
578 ], 578 ],
579− "expected": 572.2 579+ "expected": 609.3
580 } 580 }
581 ] 581 ]
582 } 582 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00123_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00123_2000", 2 "document_id": "exec_00123_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "3a8a85801b1200bb0a18063e20eb67097e3e9c34d009894a5a61963f7145608b", 5+ "document_hash": "6c868c5916de1b984d70dc327ee458814726f8ea6a1fad99f8c2fb35e7e77b7f",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 758 30+ "char_end": 631
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 "shipped_days": 9 160 "shipped_days": 9
161 } 161 }
162 ], 162 ],
163− "expected": 10163.85 163+ "expected": 10350.34
164 }, 164 },
165 { 165 {
166 "input": [ 166 "input": [
@@ -318,7 +318,7 @@
318 "shipped_days": 1 318 "shipped_days": 1
319 } 319 }
320 ], 320 ],
321− "expected": 12367.03 321+ "expected": 12553.52
322 }, 322 },
323 { 323 {
324 "input": [ 324 "input": [
@@ -404,7 +404,7 @@
404 "shipped_days": 8 404 "shipped_days": 8
405 } 405 }
406 ], 406 ],
407− "expected": 8167.73 407+ "expected": 8263.67
408 }, 408 },
409 { 409 {
410 "input": [ 410 "input": [
@@ -463,7 +463,7 @@
463 "shipped_days": 1 463 "shipped_days": 1
464 } 464 }
465 ], 465 ],
466− "expected": 5503.85 466+ "expected": 5554.96
467 }, 467 },
468 { 468 {
469 "input": [ 469 "input": [
@@ -603,7 +603,7 @@
603 "shipped_days": 11 603 "shipped_days": 11
604 } 604 }
605 ], 605 ],
606− "expected": 11804.57 606+ "expected": 11978.79
607 } 607 }
608 ] 608 ]
609 } 609 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00124_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00124_2000", 2 "document_id": "exec_00124_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "adfa94aa242540ae900f58c2db73dace6e0bb6ec243a71137685b2d527e54fdb", 5+ "document_hash": "a285cd3f7f5bbfcaa508697bf261341dc847483a97e354b95faf09bc23bb626f",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 "shipped_days": 12 97 "shipped_days": 12
98 } 98 }
99 ], 99 ],
100− "expected": 5443.0 100+ "expected": 5641.0
101 }, 101 },
102 { 102 {
103 "input": [ 103 "input": [
@@ -147,7 +147,7 @@
147 "shipped_days": 5 147 "shipped_days": 5
148 } 148 }
149 ], 149 ],
150− "expected": 5054.0 150+ "expected": 5237.0
151 }, 151 },
152 { 152 {
153 "input": [ 153 "input": [
@@ -206,7 +206,7 @@
206 "shipped_days": 12 206 "shipped_days": 12
207 } 207 }
208 ], 208 ],
209− "expected": 3318.0 209+ "expected": 3387.0
210 }, 210 },
211 { 211 {
212 "input": [ 212 "input": [
@@ -283,7 +283,7 @@
283 "shipped_days": 11 283 "shipped_days": 11
284 } 284 }
285 ], 285 ],
286− "expected": 3199.0 286+ "expected": 3369.0
287 }, 287 },
288 { 288 {
289 "input": [ 289 "input": [
@@ -369,7 +369,7 @@
369 "shipped_days": 9 369 "shipped_days": 9
370 } 370 }
371 ], 371 ],
372− "expected": 4883.0 372+ "expected": 5156.0
373 } 373 }
374 ] 374 ]
375 } 375 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00125_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00125_2000", 2 "document_id": "exec_00125_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "144a4af055228fc4045ad15b1ca9ba8229c8e5563cc4c4cf9ad94203a0072dd9", 5+ "document_hash": "386fcaff7daca426267d33af6f30657f21e4863b66d5a71d38d0b78a334eb069",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 "shipped_days": 13 97 "shipped_days": 13
98 } 98 }
99 ], 99 ],
100− "expected": 4817.6 100+ "expected": 4843.9
101 }, 101 },
102 { 102 {
103 "input": [ 103 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 5 228 "shipped_days": 5
229 } 229 }
230 ], 230 ],
231− "expected": 7490.0 231+ "expected": 7655.4
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -269,7 +269,7 @@
269 "shipped_days": 12 269 "shipped_days": 12
270 } 270 }
271 ], 271 ],
272− "expected": 2071.8 272+ "expected": 2098.1
273 }, 273 },
274 { 274 {
275 "input": [ 275 "input": [
@@ -382,7 +382,7 @@
382 "shipped_days": 11 382 "shipped_days": 11
383 } 383 }
384 ], 384 ],
385− "expected": 8298.7 385+ "expected": 8541.9
386 }, 386 },
387 { 387 {
388 "input": [ 388 "input": [
@@ -450,7 +450,7 @@
450 "shipped_days": 13 450 "shipped_days": 13
451 } 451 }
452 ], 452 ],
453− "expected": 5259.9 453+ "expected": 5536.5
454 } 454 }
455 ] 455 ]
456 } 456 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00126_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00126_2000", 2 "document_id": "exec_00126_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "a6bed1ca1c6bee001ded092712010d4dc8d3ec10d0d1f85432e8844be28b9359", 5+ "document_hash": "0729452f4b76415c15bdd0075e98c4db16464515a675ea3d8e3611adb3c8a3e9",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 6 124 "shipped_days": 6
125 } 125 }
126 ], 126 ],
127− "expected": 5475.93 127+ "expected": 5529.1
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -264,7 +264,7 @@
264 "shipped_days": 2 264 "shipped_days": 2
265 } 265 }
266 ], 266 ],
267− "expected": 11668.86 267+ "expected": 11941.25
268 }, 268 },
269 { 269 {
270 "input": [ 270 "input": [
@@ -413,7 +413,7 @@
413 "shipped_days": 1 413 "shipped_days": 1
414 } 414 }
415 ], 415 ],
416− "expected": 11668.86 416+ "expected": 11941.25
417 }, 417 },
418 { 418 {
419 "input": [ 419 "input": [
@@ -517,7 +517,7 @@
517 "shipped_days": 10 517 "shipped_days": 10
518 } 518 }
519 ], 519 ],
520− "expected": 8587.23 520+ "expected": 8696.85
521 }, 521 },
522 { 522 {
523 "input": [ 523 "input": [
@@ -630,7 +630,7 @@
630 "shipped_days": 6 630 "shipped_days": 6
631 } 631 }
632 ], 632 ],
633− "expected": 8404.43 633+ "expected": 8676.83
634 } 634 }
635 ] 635 ]
636 } 636 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00127_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00127_2000", 2 "document_id": "exec_00127_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "01e9b8b715cbfdd0d66c26d571044c18f9a6d9d2c5146b834291716879cf02d2", 5+ "document_hash": "bb6e6467042a1b7809f3ea38b83ea82e5554ca36b70042023f682efd5bae752a",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 14 133 "shipped_days": 14
134 } 134 }
135 ], 135 ],
136− "expected": 3326.0 136+ "expected": 3510.0
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 14 228 "shipped_days": 14
229 } 229 }
230 ], 230 ],
231− "expected": 283.0 231+ "expected": 296.0
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -278,7 +278,7 @@
278 "shipped_days": 14 278 "shipped_days": 14
279 } 279 }
280 ], 280 ],
281− "expected": 2936.0 281+ "expected": 3097.0
282 }, 282 },
283 { 283 {
284 "input": [ 284 "input": [
@@ -328,7 +328,7 @@
328 "shipped_days": 14 328 "shipped_days": 14
329 } 329 }
330 ], 330 ],
331− "expected": 355.0 331+ "expected": 376.0
332 }, 332 },
333 { 333 {
334 "input": [ 334 "input": [
@@ -369,7 +369,7 @@
369 "shipped_days": 12 369 "shipped_days": 12
370 } 370 }
371 ], 371 ],
372− "expected": 1798.0 372+ "expected": 1896.0
373 } 373 }
374 ] 374 ]
375 } 375 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00128_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00128_2000", 2 "document_id": "exec_00128_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "e702731f232f4b91d0513add4b301fd1ef0382eee35df22a98cef7eb727fafca", 5+ "document_hash": "ea7fe68afa60ebc43cd0a501c243c72cd3b993bfaef6e389f12308e1967e3471",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 3, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 761 30+ "char_end": 634
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -205,7 +205,7 @@
205 "shipped_days": 6 205 "shipped_days": 6
206 } 206 }
207 ], 207 ],
208− "expected": 8219.07 208+ "expected": 8595.95
209 }, 209 },
210 { 210 {
211 "input": [ 211 "input": [
@@ -291,7 +291,7 @@
291 "shipped_days": 8 291 "shipped_days": 8
292 } 292 }
293 ], 293 ],
294− "expected": 4517.31 294+ "expected": 4718.05
295 }, 295 },
296 { 296 {
297 "input": [ 297 "input": [
@@ -350,7 +350,7 @@
350 "shipped_days": 13 350 "shipped_days": 13
351 } 351 }
352 ], 352 ],
353− "expected": 2948.13 353+ "expected": 3062.55
354 }, 354 },
355 { 355 {
356 "input": [ 356 "input": [
@@ -463,7 +463,7 @@
463 "shipped_days": 8 463 "shipped_days": 8
464 } 464 }
465 ], 465 ],
466− "expected": 4934.84 466+ "expected": 5211.5
467 }, 467 },
468 { 468 {
469 "input": [ 469 "input": [
@@ -531,7 +531,7 @@
531 "shipped_days": 14 531 "shipped_days": 14
532 } 532 }
533 ], 533 ],
534− "expected": 3125.64 534+ "expected": 3300.81
535 } 535 }
536 ] 536 ]
537 } 537 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00129_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00129_2000", 2 "document_id": "exec_00129_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "e508132470ec83c8745c1899d6c954b865130cad33382718fa48d35d6bea0ec8", 5+ "document_hash": "46f3b799f18b08ee443481845e8695eff10ac2f2b23c2b177eafce3b42e2ea40",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 2 88 "shipped_days": 2
89 } 89 }
90 ], 90 ],
91− "expected": 3878.09 91+ "expected": 3976.88
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -129,7 +129,7 @@
129 "shipped_days": 2 129 "shipped_days": 2
130 } 130 }
131 ], 131 ],
132− "expected": 2759.16 132+ "expected": 2800.76
133 }, 133 },
134 { 134 {
135 "input": [ 135 "input": [
@@ -188,7 +188,7 @@
188 "shipped_days": 6 188 "shipped_days": 6
189 } 189 }
190 ], 190 ],
191− "expected": 3993.65 191+ "expected": 4137.46
192 }, 192 },
193 { 193 {
194 "input": [ 194 "input": [
@@ -346,7 +346,7 @@
346 "shipped_days": 14 346 "shipped_days": 14
347 } 347 }
348 ], 348 ],
349− "expected": 8439.66 349+ "expected": 8653.36
350 }, 350 },
351 { 351 {
352 "input": [ 352 "input": [
@@ -549,7 +549,7 @@
549 "shipped_days": 1 549 "shipped_days": 1
550 } 550 }
551 ], 551 ],
552− "expected": 10565.94 552+ "expected": 10818.88
553 } 553 }
554 ] 554 ]
555 } 555 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00130_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00130_2000", 2 "document_id": "exec_00130_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "dev", 4 "split": "dev",
5− "document_hash": "c65f14fbcb1d4d0049a6e641be2826817637866851032f95c4cc1943f87ff809", 5+ "document_hash": "c415053d25119b89bd769d02be02c390efebcd68b992b946ddf043ad29041f86",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 6 88 "shipped_days": 6
89 } 89 }
90 ], 90 ],
91− "expected": 3609.6 91+ "expected": 3633.3
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -188,7 +188,7 @@
188 "shipped_days": 2 188 "shipped_days": 2
189 } 189 }
190 ], 190 ],
191− "expected": 3823.2 191+ "expected": 3828.8
192 }, 192 },
193 { 193 {
194 "input": [ 194 "input": [
@@ -301,7 +301,7 @@
301 "shipped_days": 13 301 "shipped_days": 13
302 } 302 }
303 ], 303 ],
304− "expected": 5885.5 304+ "expected": 5944.5
305 }, 305 },
306 { 306 {
307 "input": [ 307 "input": [
@@ -378,7 +378,7 @@
378 "shipped_days": 6 378 "shipped_days": 6
379 } 379 }
380 ], 380 ],
381− "expected": 981.7 381+ "expected": 1035.1
382 } 382 }
383 ] 383 ]
384 } 384 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00201_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00201_2000", 2 "document_id": "exec_00201_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "876f090ca1801217ec4295964e7d9ce5b46b9fc96b04db7ca463d6cc8ed74978", 5+ "document_hash": "065b6c18bbd16f16126924a54946ca4bf95c5863e57b14f33e4c9d88568f8549",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 3, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 "shipped_days": 8 115 "shipped_days": 8
116 } 116 }
117 ], 117 ],
118− "expected": 2996.0 118+ "expected": 3119.0
119 }, 119 },
120 { 120 {
121 "input": [ 121 "input": [
@@ -264,7 +264,7 @@
264 "shipped_days": 1 264 "shipped_days": 1
265 } 265 }
266 ], 266 ],
267− "expected": 4291.0 267+ "expected": 4392.0
268 }, 268 },
269 { 269 {
270 "input": [ 270 "input": [
@@ -323,7 +323,7 @@
323 "shipped_days": 10 323 "shipped_days": 10
324 } 324 }
325 ], 325 ],
326− "expected": 3088.0 326+ "expected": 3164.0
327 }, 327 },
328 { 328 {
329 "input": [ 329 "input": [
@@ -373,7 +373,7 @@
373 "shipped_days": 8 373 "shipped_days": 8
374 } 374 }
375 ], 375 ],
376− "expected": 1229.0 376+ "expected": 1263.0
377 }, 377 },
378 { 378 {
379 "input": [ 379 "input": [
@@ -540,7 +540,7 @@
540 "shipped_days": 7 540 "shipped_days": 7
541 } 541 }
542 ], 542 ],
543− "expected": 4869.0 543+ "expected": 4995.0
544 } 544 }
545 ] 545 ]
546 } 546 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00202_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00202_2000", 2 "document_id": "exec_00202_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "660fc77600d7958889cef9b34f9c9a1a8c2d96ef38db054d180f225ebe3d4a42", 5+ "document_hash": "ec13194237af7efe7765a21f1d8303ab8ac449271eb48e3395906648c770bab8",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 749 30+ "char_end": 622
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 6 88 "shipped_days": 6
89 } 89 }
90 ], 90 ],
91− "expected": 5253.0 91+ "expected": 5495.0
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -219,7 +219,7 @@
219 "shipped_days": 7 219 "shipped_days": 7
220 } 220 }
221 ], 221 ],
222− "expected": 9510.0 222+ "expected": 9809.0
223 }, 223 },
224 { 224 {
225 "input": [ 225 "input": [
@@ -314,7 +314,7 @@
314 "shipped_days": 10 314 "shipped_days": 10
315 } 315 }
316 ], 316 ],
317− "expected": 4533.0 317+ "expected": 4631.0
318 }, 318 },
319 { 319 {
320 "input": [ 320 "input": [
@@ -373,7 +373,7 @@
373 "shipped_days": 5 373 "shipped_days": 5
374 } 374 }
375 ], 375 ],
376− "expected": 3891.0 376+ "expected": 3976.0
377 }, 377 },
378 { 378 {
379 "input": [ 379 "input": [
@@ -459,7 +459,7 @@
459 "shipped_days": 7 459 "shipped_days": 7
460 } 460 }
461 ], 461 ],
462− "expected": 3112.0 462+ "expected": 3118.0
463 } 463 }
464 ] 464 ]
465 } 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00203_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00203_2000", 2 "document_id": "exec_00203_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "f26e049133c3f806d2045fa255ba4e6c2594f5ecbe872414b75faaf52ceb5033", 5+ "document_hash": "39eee37c7ba5fe1b6d8fce9eef4c00e10a33d8ca049d12ae01b9521d4ac60863",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -142,7 +142,7 @@
142 "shipped_days": 6 142 "shipped_days": 6
143 } 143 }
144 ], 144 ],
145− "expected": 6045.2 145+ "expected": 6107.6
146 }, 146 },
147 { 147 {
148 "input": [ 148 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 10 228 "shipped_days": 10
229 } 229 }
230 ], 230 ],
231− "expected": 5428.4 231+ "expected": 5488.4
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -269,7 +269,7 @@
269 "shipped_days": 5 269 "shipped_days": 5
270 } 270 }
271 ], 271 ],
272− "expected": 1727.0 272+ "expected": 1748.7
273 }, 273 },
274 { 274 {
275 "input": [ 275 "input": [
@@ -373,7 +373,7 @@
373 "shipped_days": 5 373 "shipped_days": 5
374 } 374 }
375 ], 375 ],
376− "expected": 5395.2 376+ "expected": 5419.4
377 }, 377 },
378 { 378 {
379 "input": [ 379 "input": [
@@ -468,7 +468,7 @@
468 "shipped_days": 14 468 "shipped_days": 14
469 } 469 }
470 ], 470 ],
471− "expected": 4819.8 471+ "expected": 4844.0
472 } 472 }
473 ] 473 ]
474 } 474 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00204_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00204_2000", 2 "document_id": "exec_00204_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "7cc296695d97fdf8fb07de5772744af9c3a79a52f20ee1e3410187cac4dd94b2", 5+ "document_hash": "0aaf1a095160aa21c4485db3b1b171e86d9d41b904ac2a2ea102cc13df19cd22",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 2, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 9, \"min_units\": 2, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -151,7 +151,7 @@
151 "shipped_days": 14 151 "shipped_days": 14
152 } 152 }
153 ], 153 ],
154− "expected": 9785.1 154+ "expected": 10092.0
155 }, 155 },
156 { 156 {
157 "input": [ 157 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 5 228 "shipped_days": 5
229 } 229 }
230 ], 230 ],
231− "expected": 5732.4 231+ "expected": 5927.4
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -341,7 +341,7 @@
341 "shipped_days": 11 341 "shipped_days": 11
342 } 342 }
343 ], 343 ],
344− "expected": 4080.5 344+ "expected": 4275.4
345 }, 345 },
346 { 346 {
347 "input": [ 347 "input": [
@@ -382,7 +382,7 @@
382 "shipped_days": 14 382 "shipped_days": 14
383 } 383 }
384 ], 384 ],
385− "expected": 5240.0 385+ "expected": 5338.3
386 }, 386 },
387 { 387 {
388 "input": [ 388 "input": [
modified data/generated/keys/exec/exec_00205_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00205_2000", 2 "document_id": "exec_00205_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "1f9b534774ca095126415c70bc29f4400d55f083ec020a658e6ee1263a76072f", 5+ "document_hash": "c82f149ddbd77be80773d103825621fb187f06bb60d4fd570c73c2b3cd898ecb",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 "shipped_days": 13 169 "shipped_days": 13
170 } 170 }
171 ], 171 ],
172− "expected": 6545.6 172+ "expected": 6710.2
173 }, 173 },
174 { 174 {
175 "input": [ 175 "input": [
@@ -345,7 +345,7 @@
345 "shipped_days": 3 345 "shipped_days": 3
346 } 346 }
347 ], 347 ],
348− "expected": 9238.4 348+ "expected": 9422.1
349 }, 349 },
350 { 350 {
351 "input": [ 351 "input": [
@@ -557,7 +557,7 @@
557 "shipped_days": 6 557 "shipped_days": 6
558 } 558 }
559 ], 559 ],
560− "expected": 10107.0 560+ "expected": 10290.7
561 }, 561 },
562 { 562 {
563 "input": [ 563 "input": [
@@ -751,7 +751,7 @@
751 "shipped_days": 4 751 "shipped_days": 4
752 } 752 }
753 ], 753 ],
754− "expected": 10107.0 754+ "expected": 10290.7
755 }, 755 },
756 { 756 {
757 "input": [ 757 "input": [
@@ -864,7 +864,7 @@
864 "shipped_days": 5 864 "shipped_days": 5
865 } 865 }
866 ], 866 ],
867− "expected": 5911.0 867+ "expected": 5989.7
868 } 868 }
869 ] 869 ]
870 } 870 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00206_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00206_2000", 2 "document_id": "exec_00206_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "71e5bf98eb246c53babd99cdd8ec1212e594a1af3ddd2a6da69bac721ad65ddd", 5+ "document_hash": "e391c9d757bc03677c664222c3ff3f1bbab881d7f566c0072b184e0b90503339",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 8, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 759 30+ "char_end": 632
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 13 124 "shipped_days": 13
125 } 125 }
126 ], 126 ],
127− "expected": 9100.4 127+ "expected": 9357.5
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 13 228 "shipped_days": 13
229 } 229 }
230 ], 230 ],
231− "expected": 9506.1 231+ "expected": 9871.6
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -314,7 +314,7 @@
314 "shipped_days": 5 314 "shipped_days": 5
315 } 315 }
316 ], 316 ],
317− "expected": 10907.8 317+ "expected": 11222.5
318 }, 318 },
319 { 319 {
320 "input": [ 320 "input": [
@@ -364,7 +364,7 @@
364 "shipped_days": 13 364 "shipped_days": 13
365 } 365 }
366 ], 366 ],
367− "expected": 4655.5 367+ "expected": 4727.9
368 }, 368 },
369 { 369 {
370 "input": [ 370 "input": [
@@ -405,7 +405,7 @@
405 "shipped_days": 9 405 "shipped_days": 9
406 } 406 }
407 ], 407 ],
408− "expected": 3240.6 408+ "expected": 3352.1
409 } 409 }
410 ] 410 ]
411 } 411 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00207_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00207_2000", 2 "document_id": "exec_00207_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "2273befa734c54f0d780db1126ec3ce69ce8f9f829e57586a614786258681ba3", 5+ "document_hash": "7d024469b5d38776a899aa870f5af7e146953a19f1adc1d91f634cda62aa4c14",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 7 133 "shipped_days": 7
134 } 134 }
135 ], 135 ],
136− "expected": 6347.24 136+ "expected": 6534.69
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -237,7 +237,7 @@
237 "shipped_days": 2 237 "shipped_days": 2
238 } 238 }
239 ], 239 ],
240− "expected": 4633.07 240+ "expected": 4793.52
241 }, 241 },
242 { 242 {
243 "input": [ 243 "input": [
@@ -413,7 +413,7 @@
413 "shipped_days": 11 413 "shipped_days": 11
414 } 414 }
415 ], 415 ],
416− "expected": 10472.88 416+ "expected": 10704.58
417 }, 417 },
418 { 418 {
419 "input": [ 419 "input": [
@@ -634,7 +634,7 @@
634 "shipped_days": 14 634 "shipped_days": 14
635 } 635 }
636 ], 636 ],
637− "expected": 15420.92 637+ "expected": 15953.22
638 }, 638 },
639 { 639 {
640 "input": [ 640 "input": [
@@ -846,7 +846,7 @@
846 "shipped_days": 7 846 "shipped_days": 7
847 } 847 }
848 ], 848 ],
849− "expected": 15224.85 849+ "expected": 15746.83
850 } 850 }
851 ] 851 ]
852 } 852 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00208_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00208_2000", 2 "document_id": "exec_00208_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "e22e990c3c20d7feee2f70e993961e465ac68d3a867c3547dc0566739847c2d8", 5+ "document_hash": "8528b3fafcf58b9daa88ffa9d7b9b25fb6e225f9bb767849744c05510a5ace37",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 760 30+ "char_end": 633
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 8 88 "shipped_days": 8
89 } 89 }
90 ], 90 ],
91− "expected": 6763.9 91+ "expected": 7103.9
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -156,7 +156,7 @@
156 "shipped_days": 14 156 "shipped_days": 14
157 } 157 }
158 ], 158 ],
159− "expected": 4378.0 159+ "expected": 4589.7
160 }, 160 },
161 { 161 {
162 "input": [ 162 "input": [
@@ -251,7 +251,7 @@
251 "shipped_days": 5 251 "shipped_days": 5
252 } 252 }
253 ], 253 ],
254− "expected": 8918.9 254+ "expected": 9380.2
255 }, 255 },
256 { 256 {
257 "input": [ 257 "input": [
@@ -292,7 +292,7 @@
292 "shipped_days": 6 292 "shipped_days": 6
293 } 293 }
294 ], 294 ],
295− "expected": 279.1 295+ "expected": 297.1
296 }, 296 },
297 { 297 {
298 "input": [ 298 "input": [
@@ -441,7 +441,7 @@
441 "shipped_days": 9 441 "shipped_days": 9
442 } 442 }
443 ], 443 ],
444− "expected": 12672.3 444+ "expected": 13279.3
445 } 445 }
446 ] 446 ]
447 } 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00209_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00209_2000", 2 "document_id": "exec_00209_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "bd895760d266f605848302775beaceb129633108ddc74dcbe83a8642eecb0ea3", 5+ "document_hash": "27a0f4409b3611173adf72865b52c6beef014bce0282fd0498be50deeed192c9",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 7 88 "shipped_days": 7
89 } 89 }
90 ], 90 ],
91− "expected": 5042.6 91+ "expected": 5250.3
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -219,7 +219,7 @@
219 "shipped_days": 12 219 "shipped_days": 12
220 } 220 }
221 ], 221 ],
222− "expected": 7168.3 222+ "expected": 7499.5
223 }, 223 },
224 { 224 {
225 "input": [ 225 "input": [
@@ -305,7 +305,7 @@
305 "shipped_days": 14 305 "shipped_days": 14
306 } 306 }
307 ], 307 ],
308− "expected": 4988.2 308+ "expected": 5188.4
309 }, 309 },
310 { 310 {
311 "input": [ 311 "input": [
@@ -436,7 +436,7 @@
436 "shipped_days": 9 436 "shipped_days": 9
437 } 437 }
438 ], 438 ],
439− "expected": 7168.3 439+ "expected": 7499.5
440 }, 440 },
441 { 441 {
442 "input": [ 442 "input": [
@@ -531,7 +531,7 @@
531 "shipped_days": 7 531 "shipped_days": 7
532 } 532 }
533 ], 533 ],
534− "expected": 3551.1 534+ "expected": 3674.6
535 } 535 }
536 ] 536 ]
537 } 537 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00210_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00210_2000", 2 "document_id": "exec_00210_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "2678fb8184c3c22ce8bfa861af1a68ef2704dd84cb88718d63c1b668adaf1f69", 5+ "document_hash": "ffa82307f9b4986f3fc473b8b7df1e4f8f0e1c25d3b6e36cdcf4114a763427c5",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 14 88 "shipped_days": 14
89 } 89 }
90 ], 90 ],
91− "expected": 4584.59 91+ "expected": 4701.32
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -174,7 +174,7 @@
174 "shipped_days": 4 174 "shipped_days": 4
175 } 175 }
176 ], 176 ],
177− "expected": 5338.5 177+ "expected": 5502.36
178 }, 178 },
179 { 179 {
180 "input": [ 180 "input": [
@@ -296,7 +296,7 @@
296 "shipped_days": 8 296 "shipped_days": 8
297 } 297 }
298 ], 298 ],
299− "expected": 7430.25 299+ "expected": 7634.54
300 }, 300 },
301 { 301 {
302 "input": [ 302 "input": [
@@ -427,7 +427,7 @@
427 "shipped_days": 12 427 "shipped_days": 12
428 } 428 }
429 ], 429 ],
430− "expected": 8724.28 430+ "expected": 8928.56
431 }, 431 },
432 { 432 {
433 "input": [ 433 "input": [
@@ -495,7 +495,7 @@
495 "shipped_days": 8 495 "shipped_days": 8
496 } 496 }
497 ], 497 ],
498− "expected": 3752.34 498+ "expected": 3916.2
499 } 499 }
500 ] 500 ]
501 } 501 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00211_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00211_2000", 2 "document_id": "exec_00211_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "5ad4fd06e4149025eddafab6fbf5c707e06c1f7e47f3cbaec839f52cd6744e38", 5+ "document_hash": "95faa51b0302db54273e9b4469b6eae92c251463238fdad494d437435c293548",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 753 30+ "char_end": 626
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 "shipped_days": 12 169 "shipped_days": 12
170 } 170 }
171 ], 171 ],
172− "expected": 6694.0 172+ "expected": 6816.0
173 }, 173 },
174 { 174 {
175 "input": [ 175 "input": [
@@ -210,7 +210,7 @@
210 "shipped_days": 5 210 "shipped_days": 5
211 } 211 }
212 ], 212 ],
213− "expected": 2616.0 213+ "expected": 2665.0
214 }, 214 },
215 { 215 {
216 "input": [ 216 "input": [
@@ -314,7 +314,7 @@
314 "shipped_days": 1 314 "shipped_days": 1
315 } 315 }
316 ], 316 ],
317− "expected": 5873.0 317+ "expected": 5987.0
318 }, 318 },
319 { 319 {
320 "input": [ 320 "input": [
@@ -508,7 +508,7 @@
508 "shipped_days": 1 508 "shipped_days": 1
509 } 509 }
510 ], 510 ],
511− "expected": 12003.0 511+ "expected": 12354.0
512 }, 512 },
513 { 513 {
514 "input": [ 514 "input": [
modified data/generated/keys/exec/exec_00212_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00212_2000", 2 "document_id": "exec_00212_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "8a171bf76e4dc9c8921e9b2a4370fc5385ceb9695d62d19fc693110c84df06f0", 5+ "document_hash": "2be5dc8496484a99fc11a00be474f4f7f139a8f491a2ebf3fff0db8e7804dd82",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 753 30+ "char_end": 626
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -70,7 +70,7 @@
70 "shipped_days": 6 70 "shipped_days": 6
71 } 71 }
72 ], 72 ],
73− "expected": 1807.0 73+ "expected": 1814.0
74 }, 74 },
75 { 75 {
76 "input": [ 76 "input": [
@@ -156,7 +156,7 @@
156 "shipped_days": 13 156 "shipped_days": 13
157 } 157 }
158 ], 158 ],
159− "expected": 6185.0 159+ "expected": 6313.0
160 }, 160 },
161 { 161 {
162 "input": [ 162 "input": [
@@ -359,7 +359,7 @@
359 "shipped_days": 6 359 "shipped_days": 6
360 } 360 }
361 ], 361 ],
362− "expected": 12224.0 362+ "expected": 12499.0
363 }, 363 },
364 { 364 {
365 "input": [ 365 "input": [
@@ -571,7 +571,7 @@
571 "shipped_days": 10 571 "shipped_days": 10
572 } 572 }
573 ], 573 ],
574− "expected": 12154.0 574+ "expected": 12429.0
575 }, 575 },
576 { 576 {
577 "input": [ 577 "input": [
@@ -738,7 +738,7 @@
738 "shipped_days": 3 738 "shipped_days": 3
739 } 739 }
740 ], 740 ],
741− "expected": 9732.0 741+ "expected": 10002.0
742 } 742 }
743 ] 743 ]
744 } 744 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00213_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00213_2000", 2 "document_id": "exec_00213_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "55fc33fe81ccfe25e3ebdcdb70ee135b990fcf94d02bfb56e0116982ffd6df6f", 5+ "document_hash": "3c65c42333c25f1f3f90331747f08fafce263c8faa11a56d3a68bc58b2217327",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 2, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 2, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 758 30+ "char_end": 631
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 14 124 "shipped_days": 14
125 } 125 }
126 ], 126 ],
127− "expected": 7666.09 127+ "expected": 7921.14
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -210,7 +210,7 @@
210 "shipped_days": 11 210 "shipped_days": 11
211 } 211 }
212 ], 212 ],
213− "expected": 3462.69 213+ "expected": 3617.44
214 }, 214 },
215 { 215 {
216 "input": [ 216 "input": [
@@ -341,7 +341,7 @@
341 "shipped_days": 4 341 "shipped_days": 4
342 } 342 }
343 ], 343 ],
344− "expected": 6351.6 344+ "expected": 6615.95
345 }, 345 },
346 { 346 {
347 "input": [ 347 "input": [
@@ -418,7 +418,7 @@
418 "shipped_days": 13 418 "shipped_days": 13
419 } 419 }
420 ], 420 ],
421− "expected": 4539.1 421+ "expected": 4803.45
422 }, 422 },
423 { 423 {
424 "input": [ 424 "input": [
@@ -495,7 +495,7 @@
495 "shipped_days": 9 495 "shipped_days": 9
496 } 496 }
497 ], 497 ],
498− "expected": 2094.96 498+ "expected": 2181.25
499 } 499 }
500 ] 500 ]
501 } 501 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00214_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00214_2000", 2 "document_id": "exec_00214_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "0be81f6d80f5eb03ba862cbc94c8bf20b6ab1196483c947401585e7178fd458d", 5+ "document_hash": "ac0a3ee69ec7da626616f6ffb5e443a4f26a3d7857caaa20f1d4ebfa5355d28b",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 6 133 "shipped_days": 6
134 } 134 }
135 ], 135 ],
136− "expected": 9188.3 136+ "expected": 9669.7
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -264,7 +264,7 @@
264 "shipped_days": 13 264 "shipped_days": 13
265 } 265 }
266 ], 266 ],
267− "expected": 10735.2 267+ "expected": 11230.4
268 }, 268 },
269 { 269 {
270 "input": [ 270 "input": [
@@ -368,7 +368,7 @@
368 "shipped_days": 10 368 "shipped_days": 10
369 } 369 }
370 ], 370 ],
371− "expected": 6277.4 371+ "expected": 6401.6
372 }, 372 },
373 { 373 {
374 "input": [ 374 "input": [
@@ -418,7 +418,7 @@
418 "shipped_days": 4 418 "shipped_days": 4
419 } 419 }
420 ], 420 ],
421− "expected": 6555.5 421+ "expected": 6789.8
422 }, 422 },
423 { 423 {
424 "input": [ 424 "input": [
@@ -486,7 +486,7 @@
486 "shipped_days": 14 486 "shipped_days": 14
487 } 487 }
488 ], 488 ],
489− "expected": 7305.1 489+ "expected": 7485.3
490 } 490 }
491 ] 491 ]
492 } 492 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00215_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00215_2000", 2 "document_id": "exec_00215_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "37b95a047621c2a78beaff095b27647064c994bf6ede6ef6b3e91f8296b70eff", 5+ "document_hash": "a4094c70181359b8442ead384be9497e3fb0c46a4d9266cce07ca77bd9e26e2b",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 5, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 5, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 753 30+ "char_end": 626
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 2 124 "shipped_days": 2
125 } 125 }
126 ], 126 ],
127− "expected": 7073.4 127+ "expected": 7469.6
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 9 228 "shipped_days": 9
229 } 229 }
230 ], 230 ],
231− "expected": 12236.6 231+ "expected": 12753.4
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -431,7 +431,7 @@
431 "shipped_days": 6 431 "shipped_days": 6
432 } 432 }
433 ], 433 ],
434− "expected": 19490.8 434+ "expected": 20372.7
435 }, 435 },
436 { 436 {
437 "input": [ 437 "input": [
@@ -598,7 +598,7 @@
598 "shipped_days": 5 598 "shipped_days": 5
599 } 599 }
600 ], 600 ],
601− "expected": 15129.1 601+ "expected": 15765.2
602 }, 602 },
603 { 603 {
604 "input": [ 604 "input": [
@@ -729,7 +729,7 @@
729 "shipped_days": 8 729 "shipped_days": 8
730 } 730 }
731 ], 731 ],
732− "expected": 13158.9 732+ "expected": 13686.7
733 } 733 }
734 ] 734 ]
735 } 735 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00216_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00216_2000", 2 "document_id": "exec_00216_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "1deddf301b0b2291cb9bdc4213848eceb17ece12032a2e454bcceb4ca4174397", 5+ "document_hash": "1a218de32e6fe4bbe97927ea16330bda3b598ed66c1517824d94ff90f944e802",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 756 30+ "char_end": 629
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -142,7 +142,7 @@
142 "shipped_days": 9 142 "shipped_days": 9
143 } 143 }
144 ], 144 ],
145− "expected": 5896.65 145+ "expected": 6072.0
146 }, 146 },
147 { 147 {
148 "input": [ 148 "input": [
@@ -264,7 +264,7 @@
264 "shipped_days": 6 264 "shipped_days": 6
265 } 265 }
266 ], 266 ],
267− "expected": 7365.6 267+ "expected": 7600.1
268 }, 268 },
269 { 269 {
270 "input": [ 270 "input": [
@@ -458,7 +458,7 @@
458 "shipped_days": 10 458 "shipped_days": 10
459 } 459 }
460 ], 460 ],
461− "expected": 8732.34 461+ "expected": 9064.48
462 }, 462 },
463 { 463 {
464 "input": [ 464 "input": [
@@ -526,7 +526,7 @@
526 "shipped_days": 14 526 "shipped_days": 14
527 } 527 }
528 ], 528 ],
529− "expected": 2062.77 529+ "expected": 2197.03
530 }, 530 },
531 { 531 {
532 "input": [ 532 "input": [
@@ -711,7 +711,7 @@
711 "shipped_days": 10 711 "shipped_days": 10
712 } 712 }
713 ], 713 ],
714− "expected": 8732.34 714+ "expected": 9064.48
715 } 715 }
716 ] 716 ]
717 } 717 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00217_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00217_2000", 2 "document_id": "exec_00217_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "7f71cb12d994e6fe2646a0f7b337c441f923868ec6b21c29533122fda4d87a73", 5+ "document_hash": "1223c7391b1da1aeae2d242c0e41de4639218cfe0c90ba9065f1cce42f0eedbd",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 753 30+ "char_end": 626
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 10 133 "shipped_days": 10
134 } 134 }
135 ], 135 ],
136− "expected": 6373.0 136+ "expected": 6665.0
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -201,7 +201,7 @@
201 "shipped_days": 13 201 "shipped_days": 13
202 } 202 }
203 ], 203 ],
204− "expected": 4341.0 204+ "expected": 4602.0
205 }, 205 },
206 { 206 {
207 "input": [ 207 "input": [
@@ -350,7 +350,7 @@
350 "shipped_days": 11 350 "shipped_days": 11
351 } 351 }
352 ], 352 ],
353− "expected": 11700.0 353+ "expected": 12228.0
354 }, 354 },
355 { 355 {
356 "input": [ 356 "input": [
@@ -562,7 +562,7 @@
562 "shipped_days": 5 562 "shipped_days": 5
563 } 563 }
564 ], 564 ],
565− "expected": 16809.0 565+ "expected": 17574.0
566 }, 566 },
567 { 567 {
568 "input": [ 568 "input": [
@@ -693,7 +693,7 @@
693 "shipped_days": 4 693 "shipped_days": 4
694 } 694 }
695 ], 695 ],
696− "expected": 9406.0 696+ "expected": 9766.0
697 } 697 }
698 ] 698 ]
699 } 699 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00218_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00218_2000", 2 "document_id": "exec_00218_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "2a77fd38142317b7e8868dfcaebb42248ddd462e9339268378a26e4b587eab3d", 5+ "document_hash": "971d94c4e7b4ae02142ffb6eb92af9d2cc0f6e4d52e94a64c5132b510197ce8a",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 752 30+ "char_end": 625
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -187,7 +187,7 @@
187 "shipped_days": 9 187 "shipped_days": 9
188 } 188 }
189 ], 189 ],
190− "expected": 8482.0 190+ "expected": 8709.0
191 }, 191 },
192 { 192 {
193 "input": [ 193 "input": [
@@ -255,7 +255,7 @@
255 "shipped_days": 9 255 "shipped_days": 9
256 } 256 }
257 ], 257 ],
258− "expected": 1995.0 258+ "expected": 2060.0
259 }, 259 },
260 { 260 {
261 "input": [ 261 "input": [
@@ -350,7 +350,7 @@
350 "shipped_days": 3 350 "shipped_days": 3
351 } 351 }
352 ], 352 ],
353− "expected": 3350.0 353+ "expected": 3392.0
354 }, 354 },
355 { 355 {
356 "input": [ 356 "input": [
@@ -490,7 +490,7 @@
490 "shipped_days": 6 490 "shipped_days": 6
491 } 491 }
492 ], 492 ],
493− "expected": 5406.0 493+ "expected": 5568.0
494 }, 494 },
495 { 495 {
496 "input": [ 496 "input": [
@@ -648,7 +648,7 @@
648 "shipped_days": 10 648 "shipped_days": 10
649 } 649 }
650 ], 650 ],
651− "expected": 8482.0 651+ "expected": 8709.0
652 } 652 }
653 ] 653 ]
654 } 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00219_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00219_2000", 2 "document_id": "exec_00219_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "501d502ac1cd6920ddaeccbc4f0c925ddf4ca966cfb601ffd9d0547d97a741ed", 5+ "document_hash": "390edd848add1a4cf011660e8ac9a6a6aa381c4508a8f68635f3217ce20979cb",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 5, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 5, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 759 30+ "char_end": 632
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -187,7 +187,7 @@
187 "shipped_days": 12 187 "shipped_days": 12
188 } 188 }
189 ], 189 ],
190− "expected": 9319.14 190+ "expected": 9797.23
191 }, 191 },
192 { 192 {
193 "input": [ 193 "input": [
@@ -381,7 +381,7 @@
381 "shipped_days": 11 381 "shipped_days": 11
382 } 382 }
383 ], 383 ],
384− "expected": 15430.66 384+ "expected": 16029.27
385 }, 385 },
386 { 386 {
387 "input": [ 387 "input": [
@@ -503,7 +503,7 @@
503 "shipped_days": 14 503 "shipped_days": 14
504 } 504 }
505 ], 505 ],
506− "expected": 13562.0 506+ "expected": 14131.7
507 }, 507 },
508 { 508 {
509 "input": [ 509 "input": [
@@ -652,7 +652,7 @@
652 "shipped_days": 6 652 "shipped_days": 6
653 } 653 }
654 ], 654 ],
655− "expected": 15200.05 655+ "expected": 15771.11
656 }, 656 },
657 { 657 {
658 "input": [ 658 "input": [
@@ -711,7 +711,7 @@
711 "shipped_days": 6 711 "shipped_days": 6
712 } 712 }
713 ], 713 ],
714− "expected": 6985.93 714+ "expected": 7345.77
715 } 715 }
716 ] 716 ]
717 } 717 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00220_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00220_2000", 2 "document_id": "exec_00220_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "a7487c2eb12039fcc44c64460641febdd19eaf0e13d8b6b8f00d018ca564a8b6", 5+ "document_hash": "31eca9e7afc70ac5a795edc1d1529c531e0d028342802a61f4496c013e13bf45",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 759 30+ "char_end": 632
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -205,7 +205,7 @@
205 "shipped_days": 12 205 "shipped_days": 12
206 } 206 }
207 ], 207 ],
208− "expected": 10109.6 208+ "expected": 10644.1
209 }, 209 },
210 { 210 {
211 "input": [ 211 "input": [
@@ -309,7 +309,7 @@
309 "shipped_days": 3 309 "shipped_days": 3
310 } 310 }
311 ], 311 ],
312− "expected": 3859.8 312+ "expected": 4025.2
313 }, 313 },
314 { 314 {
315 "input": [ 315 "input": [
@@ -449,7 +449,7 @@
449 "shipped_days": 3 449 "shipped_days": 3
450 } 450 }
451 ], 451 ],
452− "expected": 9322.5 452+ "expected": 9846.6
453 }, 453 },
454 { 454 {
455 "input": [ 455 "input": [
@@ -526,7 +526,7 @@
526 "shipped_days": 1 526 "shipped_days": 1
527 } 527 }
528 ], 528 ],
529− "expected": 4911.7 529+ "expected": 5199.9
530 }, 530 },
531 { 531 {
532 "input": [ 532 "input": [
@@ -639,7 +639,7 @@
639 "shipped_days": 8 639 "shipped_days": 8
640 } 640 }
641 ], 641 ],
642− "expected": 6462.4 642+ "expected": 6789.1
643 } 643 }
644 ] 644 ]
645 } 645 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00221_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00221_2000", 2 "document_id": "exec_00221_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "a986eec8e76aa658fd694272657b8e201d940be0671e7769a934ffe76ca4748a", 5+ "document_hash": "e228dfaa0d04173ee28b43513e5fc06914edf5d0c02b09bf0fc534531a583669",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 760 30+ "char_end": 633
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 "shipped_days": 10 115 "shipped_days": 10
116 } 116 }
117 ], 117 ],
118− "expected": 6742.98 118+ "expected": 6948.54
119 }, 119 },
120 { 120 {
121 "input": [ 121 "input": [
@@ -165,7 +165,7 @@
165 "shipped_days": 5 165 "shipped_days": 5
166 } 166 }
167 ], 167 ],
168− "expected": 2437.55 168+ "expected": 2557.8
169 }, 169 },
170 { 170 {
171 "input": [ 171 "input": [
@@ -287,7 +287,7 @@
287 "shipped_days": 10 287 "shipped_days": 10
288 } 288 }
289 ], 289 ],
290− "expected": 7289.29 290+ "expected": 7515.78
291 }, 291 },
292 { 292 {
293 "input": [ 293 "input": [
@@ -355,7 +355,7 @@
355 "shipped_days": 5 355 "shipped_days": 5
356 } 356 }
357 ], 357 ],
358− "expected": 3623.37 358+ "expected": 3736.67
359 }, 359 },
360 { 360 {
361 "input": [ 361 "input": [
@@ -423,7 +423,7 @@
423 "shipped_days": 8 423 "shipped_days": 8
424 } 424 }
425 ], 425 ],
426− "expected": 2195.08 426+ "expected": 2213.17
427 } 427 }
428 ] 428 ]
429 } 429 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00222_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00222_2000", 2 "document_id": "exec_00222_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "fa31c738b8b80170524adedf048965eb739a7ebf7de9fcd4f4192d331dcc3346", 5+ "document_hash": "040260f25a42680fcc038386368506adaa004bc2592e75e3fbb5903b54cc19ab",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 750 30+ "char_end": 623
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 "shipped_days": 4 124 "shipped_days": 4
125 } 125 }
126 ], 126 ],
127− "expected": 6567.0 127+ "expected": 6895.0
128 }, 128 },
129 { 129 {
130 "input": [ 130 "input": [
@@ -282,7 +282,7 @@
282 "shipped_days": 1 282 "shipped_days": 1
283 } 283 }
284 ], 284 ],
285− "expected": 11349.0 285+ "expected": 11712.0
286 }, 286 },
287 { 287 {
288 "input": [ 288 "input": [
@@ -422,7 +422,7 @@
422 "shipped_days": 14 422 "shipped_days": 14
423 } 423 }
424 ], 424 ],
425− "expected": 10867.0 425+ "expected": 11226.0
426 }, 426 },
427 { 427 {
428 "input": [ 428 "input": [
@@ -526,7 +526,7 @@
526 "shipped_days": 11 526 "shipped_days": 11
527 } 527 }
528 ], 528 ],
529− "expected": 5274.0 529+ "expected": 5511.0
530 }, 530 },
531 { 531 {
532 "input": [ 532 "input": [
@@ -576,7 +576,7 @@
576 "shipped_days": 6 576 "shipped_days": 6
577 } 577 }
578 ], 578 ],
579− "expected": 1237.0 579+ "expected": 1268.0
580 } 580 }
581 ] 581 ]
582 } 582 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00223_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00223_2000", 2 "document_id": "exec_00223_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "9b2d718a88e7b451a6e6d433e81fa6c3862777588f40791fe888b2d7a44efa42", 5+ "document_hash": "7eef13486249d89cf05bda367ea9dae2ce2479b2236c1917b2f067c49a981619",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -214,7 +214,7 @@
214 "shipped_days": 1 214 "shipped_days": 1
215 } 215 }
216 ], 216 ],
217− "expected": 12157.0 217+ "expected": 12469.0
218 }, 218 },
219 { 219 {
220 "input": [ 220 "input": [
@@ -309,7 +309,7 @@
309 "shipped_days": 8 309 "shipped_days": 8
310 } 310 }
311 ], 311 ],
312− "expected": 3305.0 312+ "expected": 3371.0
313 }, 313 },
314 { 314 {
315 "input": [ 315 "input": [
@@ -494,7 +494,7 @@
494 "shipped_days": 9 494 "shipped_days": 9
495 } 495 }
496 ], 496 ],
497− "expected": 10095.0 497+ "expected": 10407.0
498 }, 498 },
499 { 499 {
500 "input": [ 500 "input": [
@@ -562,7 +562,7 @@
562 "shipped_days": 9 562 "shipped_days": 9
563 } 563 }
564 ], 564 ],
565− "expected": 5109.0 565+ "expected": 5263.0
566 }, 566 },
567 { 567 {
568 "input": [ 568 "input": [
@@ -720,7 +720,7 @@
720 "shipped_days": 1 720 "shipped_days": 1
721 } 721 }
722 ], 722 ],
723− "expected": 10891.0 723+ "expected": 11167.0
724 } 724 }
725 ] 725 ]
726 } 726 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00224_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00224_2000", 2 "document_id": "exec_00224_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "526a53c484a99dde8d83b8fc83d103060128d6331659920d7688b8836de1ec53", 5+ "document_hash": "3e642abb57841365b827053f2437a250dcb419539389820bac778d4596c064fd",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 760 30+ "char_end": 633
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 "shipped_days": 12 106 "shipped_days": 12
107 } 107 }
108 ], 108 ],
109− "expected": 5224.44 109+ "expected": 5450.26
110 }, 110 },
111 { 111 {
112 "input": [ 112 "input": [
@@ -165,7 +165,7 @@
165 "shipped_days": 4 165 "shipped_days": 4
166 } 166 }
167 ], 167 ],
168− "expected": 1529.84 168+ "expected": 1612.2
169 }, 169 },
170 { 170 {
171 "input": [ 171 "input": [
@@ -287,7 +287,7 @@
287 "shipped_days": 8 287 "shipped_days": 8
288 } 288 }
289 ], 289 ],
290− "expected": 7160.51 290+ "expected": 7528.74
291 }, 291 },
292 { 292 {
293 "input": [ 293 "input": [
@@ -364,7 +364,7 @@
364 "shipped_days": 11 364 "shipped_days": 11
365 } 365 }
366 ], 366 ],
367− "expected": 3959.61 367+ "expected": 4149.28
368 }, 368 },
369 { 369 {
370 "input": [ 370 "input": [
@@ -423,7 +423,7 @@
423 "shipped_days": 6 423 "shipped_days": 6
424 } 424 }
425 ], 425 ],
426− "expected": 2005.6 426+ "expected": 2086.3
427 } 427 }
428 ] 428 ]
429 } 429 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00225_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00225_2000", 2 "document_id": "exec_00225_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "0d5d08bcef45a939605c15cb0d0b5b1ac3c2304c8358dae05fa7da75db07621f", 5+ "document_hash": "54918c16a4ae3706a87d182112fca2a39c82c77e3a60fd25a8b47b958f2a6b21",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 "shipped_days": 13 79 "shipped_days": 13
80 } 80 }
81 ], 81 ],
82− "expected": 2122.0 82+ "expected": 2201.0
83 }, 83 },
84 { 84 {
85 "input": [ 85 "input": [
@@ -174,7 +174,7 @@
174 "shipped_days": 10 174 "shipped_days": 10
175 } 175 }
176 ], 176 ],
177− "expected": 3922.0 177+ "expected": 4123.0
178 }, 178 },
179 { 179 {
180 "input": [ 180 "input": [
@@ -251,7 +251,7 @@
251 "shipped_days": 12 251 "shipped_days": 12
252 } 252 }
253 ], 253 ],
254− "expected": 1266.0 254+ "expected": 1352.0
255 }, 255 },
256 { 256 {
257 "input": [ 257 "input": [
@@ -319,7 +319,7 @@
319 "shipped_days": 1 319 "shipped_days": 1
320 } 320 }
321 ], 321 ],
322− "expected": 3208.0 322+ "expected": 3364.0
323 }, 323 },
324 { 324 {
325 "input": [ 325 "input": [
@@ -414,7 +414,7 @@
414 "shipped_days": 13 414 "shipped_days": 13
415 } 415 }
416 ], 416 ],
417− "expected": 3922.0 417+ "expected": 4123.0
418 } 418 }
419 ] 419 ]
420 } 420 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00226_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00226_2000", 2 "document_id": "exec_00226_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "32ca9297d301e1956917680ad370eda2add0304f9742c49a2023cfa3f2a82bd2", 5+ "document_hash": "7a73067df93b17aba1a608bce48f7a6ff16d480af8d23713ec889405f279fc7d",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 750 30+ "char_end": 623
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 "shipped_days": 9 106 "shipped_days": 9
107 } 107 }
108 ], 108 ],
109− "expected": 6307.0 109+ "expected": 6505.0
110 }, 110 },
111 { 111 {
112 "input": [ 112 "input": [
@@ -156,7 +156,7 @@
156 "shipped_days": 7 156 "shipped_days": 7
157 } 157 }
158 ], 158 ],
159− "expected": 1138.0 159+ "expected": 1182.0
160 }, 160 },
161 { 161 {
162 "input": [ 162 "input": [
@@ -224,7 +224,7 @@
224 "shipped_days": 3 224 "shipped_days": 3
225 } 225 }
226 ], 226 ],
227− "expected": 3931.0 227+ "expected": 4064.0
228 }, 228 },
229 { 229 {
230 "input": [ 230 "input": [
@@ -337,7 +337,7 @@
337 "shipped_days": 3 337 "shipped_days": 3
338 } 338 }
339 ], 339 ],
340− "expected": 8074.0 340+ "expected": 8340.0
341 }, 341 },
342 { 342 {
343 "input": [ 343 "input": [
@@ -486,7 +486,7 @@
486 "shipped_days": 14 486 "shipped_days": 14
487 } 487 }
488 ], 488 ],
489− "expected": 7468.0 489+ "expected": 7645.0
490 } 490 }
491 ] 491 ]
492 } 492 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00227_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00227_2000", 2 "document_id": "exec_00227_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "0297d0253d8ecb7ba91156d4221aff1d54ba5efaed7fe7664734e63d44d31bd0", 5+ "document_hash": "6afe19e10d57be0082db797187045b004680fa22a13a56a7b8f7e2a51b40597d",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 3, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 751 30+ "char_end": 624
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -70,7 +70,7 @@
70 "shipped_days": 1 70 "shipped_days": 1
71 } 71 }
72 ], 72 ],
73− "expected": 3306.5 73+ "expected": 3366.2
74 }, 74 },
75 { 75 {
76 "input": [ 76 "input": [
@@ -138,7 +138,7 @@
138 "shipped_days": 10 138 "shipped_days": 10
139 } 139 }
140 ], 140 ],
141− "expected": 5976.1 141+ "expected": 6181.3
142 }, 142 },
143 { 143 {
144 "input": [ 144 "input": [
@@ -242,7 +242,7 @@
242 "shipped_days": 8 242 "shipped_days": 8
243 } 243 }
244 ], 244 ],
245− "expected": 5590.1 245+ "expected": 5863.7
246 }, 246 },
247 { 247 {
248 "input": [ 248 "input": [
@@ -346,7 +346,7 @@
346 "shipped_days": 3 346 "shipped_days": 3
347 } 347 }
348 ], 348 ],
349− "expected": 7501.9 349+ "expected": 7775.4
350 }, 350 },
351 { 351 {
352 "input": [ 352 "input": [
@@ -387,7 +387,7 @@
387 "shipped_days": 14 387 "shipped_days": 14
388 } 388 }
389 ], 389 ],
390− "expected": 4197.5 390+ "expected": 4333.5
391 } 391 }
392 ] 392 ]
393 } 393 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00228_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00228_2000", 2 "document_id": "exec_00228_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "7bd8eb7ac7409f89d5f44f1f5001fd0660aece03d12d92588e821c86e0d1f69b", 5+ "document_hash": "08d5c40fa24cccbcccaac14d408c241ab602604a712b727f12f570eb56887a74",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 5, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 5, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 751 30+ "char_end": 624
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 "shipped_days": 1 115 "shipped_days": 1
116 } 116 }
117 ], 117 ],
118− "expected": 4217.76 118+ "expected": 4336.63
119 }, 119 },
120 { 120 {
121 "input": [ 121 "input": [
@@ -165,7 +165,7 @@
165 "shipped_days": 10 165 "shipped_days": 10
166 } 166 }
167 ], 167 ],
168− "expected": 1667.06 168+ "expected": 1740.67
169 }, 169 },
170 { 170 {
171 "input": [ 171 "input": [
@@ -296,7 +296,7 @@
296 "shipped_days": 5 296 "shipped_days": 5
297 } 297 }
298 ], 298 ],
299− "expected": 6250.25 299+ "expected": 6433.51
300 }, 300 },
301 { 301 {
302 "input": [ 302 "input": [
@@ -391,7 +391,7 @@
391 "shipped_days": 5 391 "shipped_days": 5
392 } 392 }
393 ], 393 ],
394− "expected": 6440.03 394+ "expected": 6612.64
395 }, 395 },
396 { 396 {
397 "input": [ 397 "input": [
@@ -603,7 +603,7 @@
603 "shipped_days": 1 603 "shipped_days": 1
604 } 604 }
605 ], 605 ],
606− "expected": 10273.51 606+ "expected": 10576.4
607 } 607 }
608 ] 608 ]
609 } 609 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00229_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00229_2000", 2 "document_id": "exec_00229_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "488c5efd60f8e64b88269226071b8befe058a5e2442b99934157b38f5fc41a19", 5+ "document_hash": "b6a34443a51bb95703036fc2467d0c50ad248f99f3c11c0a35d6bc7fcf0ad481",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 2 133 "shipped_days": 2
134 } 134 }
135 ], 135 ],
136− "expected": 7117.0 136+ "expected": 7480.0
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -183,7 +183,7 @@
183 "shipped_days": 12 183 "shipped_days": 12
184 } 184 }
185 ], 185 ],
186− "expected": 2978.0 186+ "expected": 3031.0
187 }, 187 },
188 { 188 {
189 "input": [ 189 "input": [
@@ -224,7 +224,7 @@
224 "shipped_days": 2 224 "shipped_days": 2
225 } 225 }
226 ], 226 ],
227− "expected": 5195.0 227+ "expected": 5404.0
228 }, 228 },
229 { 229 {
230 "input": [ 230 "input": [
@@ -292,7 +292,7 @@
292 "shipped_days": 6 292 "shipped_days": 6
293 } 293 }
294 ], 294 ],
295− "expected": 6772.0 295+ "expected": 7044.0
296 }, 296 },
297 { 297 {
298 "input": [ 298 "input": [
@@ -432,7 +432,7 @@
432 "shipped_days": 7 432 "shipped_days": 7
433 } 433 }
434 ], 434 ],
435− "expected": 11561.0 435+ "expected": 12047.0
436 } 436 }
437 ] 437 ]
438 } 438 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00230_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00230_2000", 2 "document_id": "exec_00230_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "2e09e5c09991cb1b7c5c288b9a49bb058042a17dc7fd32cde98e2b705477598c", 5+ "document_hash": "cbb9dba4a75d2570390205536d928042eae0341c49e7dcc11a2cea988c175ba9",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -196,7 +196,7 @@
196 "shipped_days": 3 196 "shipped_days": 3
197 } 197 }
198 ], 198 ],
199− "expected": 9688.0 199+ "expected": 9995.0
200 }, 200 },
201 { 201 {
202 "input": [ 202 "input": [
@@ -273,7 +273,7 @@
273 "shipped_days": 7 273 "shipped_days": 7
274 } 274 }
275 ], 275 ],
276− "expected": 3730.0 276+ "expected": 3834.0
277 }, 277 },
278 { 278 {
279 "input": [ 279 "input": [
@@ -368,7 +368,7 @@
368 "shipped_days": 13 368 "shipped_days": 13
369 } 369 }
370 ], 370 ],
371− "expected": 10447.0 371+ "expected": 10628.0
372 }, 372 },
373 { 373 {
374 "input": [ 374 "input": [
@@ -499,7 +499,7 @@
499 "shipped_days": 13 499 "shipped_days": 13
500 } 500 }
501 ], 501 ],
502− "expected": 7718.0 502+ "expected": 7815.0
503 }, 503 },
504 { 504 {
505 "input": [ 505 "input": [
@@ -612,7 +612,7 @@
612 "shipped_days": 10 612 "shipped_days": 10
613 } 613 }
614 ], 614 ],
615− "expected": 7101.0 615+ "expected": 7378.0
616 } 616 }
617 ] 617 ]
618 } 618 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00231_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00231_2000", 2 "document_id": "exec_00231_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "9071b939e284c02834a84c70b2af1783b2a26d6217aa2184b3224f0c27400eaf", 5+ "document_hash": "b599ec4f6ea744baef17b36d05ab828f105675eeebf81a780befd49f389ec9d3",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 754 30+ "char_end": 627
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 "shipped_days": 2 133 "shipped_days": 2
134 } 134 }
135 ], 135 ],
136− "expected": 4152.7 136+ "expected": 4283.8
137 }, 137 },
138 { 138 {
139 "input": [ 139 "input": [
@@ -210,7 +210,7 @@
210 "shipped_days": 7 210 "shipped_days": 7
211 } 211 }
212 ], 212 ],
213− "expected": 4440.6 213+ "expected": 4715.0
214 }, 214 },
215 { 215 {
216 "input": [ 216 "input": [
@@ -377,7 +377,7 @@
377 "shipped_days": 9 377 "shipped_days": 9
378 } 378 }
379 ], 379 ],
380− "expected": 9136.0 380+ "expected": 9560.7
381 }, 381 },
382 { 382 {
383 "input": [ 383 "input": [
@@ -481,7 +481,7 @@
481 "shipped_days": 7 481 "shipped_days": 7
482 } 482 }
483 ], 483 ],
484− "expected": 5626.6 484+ "expected": 5933.4
485 }, 485 },
486 { 486 {
487 "input": [ 487 "input": [
@@ -648,7 +648,7 @@
648 "shipped_days": 1 648 "shipped_days": 1
649 } 649 }
650 ], 650 ],
651− "expected": 9136.0 651+ "expected": 9560.7
652 } 652 }
653 ] 653 ]
654 } 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00232_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00232_2000", 2 "document_id": "exec_00232_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "6bb55261f58b559a6304971baddf54cabc7de9843d30afe350a659e21fa02b73", 5+ "document_hash": "94e54bf28dac5a404fdf94451d3e632c18f63dff001cc758f89f5239d27f659b",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 3, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 2, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 3, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 2, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 "shipped_days": 9 97 "shipped_days": 9
98 } 98 }
99 ], 99 ],
100− "expected": 2607.84 100+ "expected": 2758.36
101 }, 101 },
102 { 102 {
103 "input": [ 103 "input": [
@@ -219,7 +219,7 @@
219 "shipped_days": 11 219 "shipped_days": 11
220 } 220 }
221 ], 221 ],
222− "expected": 5806.2 222+ "expected": 6158.4
223 }, 223 },
224 { 224 {
225 "input": [ 225 "input": [
@@ -332,7 +332,7 @@
332 "shipped_days": 10 332 "shipped_days": 10
333 } 333 }
334 ], 334 ],
335− "expected": 5434.24 335+ "expected": 5765.79
336 }, 336 },
337 { 337 {
338 "input": [ 338 "input": [
@@ -481,7 +481,7 @@
481 "shipped_days": 9 481 "shipped_days": 9
482 } 482 }
483 ], 483 ],
484− "expected": 5806.2 484+ "expected": 6158.4
485 }, 485 },
486 { 486 {
487 "input": [ 487 "input": [
@@ -522,7 +522,7 @@
522 "shipped_days": 10 522 "shipped_days": 10
523 } 523 }
524 ], 524 ],
525− "expected": 2784.23 525+ "expected": 2949.0
526 } 526 }
527 ] 527 ]
528 } 528 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00233_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00233_2000", 2 "document_id": "exec_00233_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "f88eb8610ee0ae9a0cc4c953e6f302bceb7e6a9f131c2e5f775a4026d048e997", 5+ "document_hash": "c0f56e7b9b73df623955c8b6822884e4b53df031c58736213f498ed2b4d693dc",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 9, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 750 30+ "char_end": 623
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 "shipped_days": 6 160 "shipped_days": 6
161 } 161 }
162 ], 162 ],
163− "expected": 10002.2 163+ "expected": 10256.6
164 }, 164 },
165 { 165 {
166 "input": [ 166 "input": [
@@ -273,7 +273,7 @@
273 "shipped_days": 3 273 "shipped_days": 3
274 } 274 }
275 ], 275 ],
276− "expected": 9483.9 276+ "expected": 9650.7
277 }, 277 },
278 { 278 {
279 "input": [ 279 "input": [
@@ -436,7 +436,7 @@
436 "shipped_days": 9 436 "shipped_days": 9
437 } 437 }
438 ], 438 ],
439− "expected": 9655.3 439+ "expected": 9888.2
440 }, 440 },
441 { 441 {
442 "input": [ 442 "input": [
@@ -513,7 +513,7 @@
513 "shipped_days": 6 513 "shipped_days": 6
514 } 514 }
515 ], 515 ],
516− "expected": 5501.4 516+ "expected": 5662.6
517 } 517 }
518 ] 518 ]
519 } 519 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00234_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00234_2000", 2 "document_id": "exec_00234_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "89b49129e4581b4b424c4986c040ee62a62ff6f482d53151d96cbcf944c97e02", 5+ "document_hash": "3522724b66617b60e55fa67efc23998ff528130352e7a960ee1e052d697670a5",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 753 30+ "char_end": 626
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -205,7 +205,7 @@
205 "shipped_days": 8 205 "shipped_days": 8
206 } 206 }
207 ], 207 ],
208− "expected": 6977.4 208+ "expected": 7260.4
209 }, 209 },
210 { 210 {
211 "input": [ 211 "input": [
@@ -372,7 +372,7 @@
372 "shipped_days": 6 372 "shipped_days": 6
373 } 373 }
374 ], 374 ],
375− "expected": 6926.7 375+ "expected": 7206.5
376 }, 376 },
377 { 377 {
378 "input": [ 378 "input": [
@@ -494,7 +494,7 @@
494 "shipped_days": 1 494 "shipped_days": 1
495 } 495 }
496 ], 496 ],
497− "expected": 4747.4 497+ "expected": 4987.8
498 }, 498 },
499 { 499 {
500 "input": [ 500 "input": [
@@ -553,7 +553,7 @@
553 "shipped_days": 6 553 "shipped_days": 6
554 } 554 }
555 ], 555 ],
556− "expected": 3576.5 556+ "expected": 3677.4
557 }, 557 },
558 { 558 {
559 "input": [ 559 "input": [
@@ -693,7 +693,7 @@
693 "shipped_days": 6 693 "shipped_days": 6
694 } 694 }
695 ], 695 ],
696− "expected": 5237.7 696+ "expected": 5505.7
697 } 697 }
698 ] 698 ]
699 } 699 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00235_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00235_2000", 2 "document_id": "exec_00235_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "7870795b3e6c98bee59eb42d0e036ccc1d88fe696a6e7c94016c77fb0f3cefe4", 5+ "document_hash": "20bfca2f1ed7277720116b37988eb2bf9331505c61f7491ff7f131ed0b2d70e1",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 750 30+ "char_end": 623
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 "shipped_days": 7 106 "shipped_days": 7
107 } 107 }
108 ], 108 ],
109− "expected": 4951.0 109+ "expected": 5095.0
110 }, 110 },
111 { 111 {
112 "input": [ 112 "input": [
@@ -174,7 +174,7 @@
174 "shipped_days": 2 174 "shipped_days": 2
175 } 175 }
176 ], 176 ],
177− "expected": 2359.0 177+ "expected": 2434.0
178 }, 178 },
179 { 179 {
180 "input": [ 180 "input": [
@@ -278,7 +278,7 @@
278 "shipped_days": 2 278 "shipped_days": 2
279 } 279 }
280 ], 280 ],
281− "expected": 3596.0 281+ "expected": 3751.0
282 }, 282 },
283 { 283 {
284 "input": [ 284 "input": [
@@ -337,7 +337,7 @@
337 "shipped_days": 1 337 "shipped_days": 1
338 } 338 }
339 ], 339 ],
340− "expected": 1146.0 340+ "expected": 1213.0
341 }, 341 },
342 { 342 {
343 "input": [ 343 "input": [
@@ -441,7 +441,7 @@
441 "shipped_days": 11 441 "shipped_days": 11
442 } 442 }
443 ], 443 ],
444− "expected": 4940.0 444+ "expected": 5083.0
445 } 445 }
446 ] 446 ]
447 } 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00236_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00236_2000", 2 "document_id": "exec_00236_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "dabe94079fad1aa1fef7653ff5583522be249aa0c33c58334ae1d261091d83f9", 5+ "document_hash": "23053422d45849043d9eb3706daf105a581cd66b69e96220ab1285c7c92b2058",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 9.975}", 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 9.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 757 30+ "char_end": 630
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 "shipped_days": 2 115 "shipped_days": 2
116 } 116 }
117 ], 117 ],
118− "expected": 5423.58 118+ "expected": 5614.82
119 }, 119 },
120 { 120 {
121 "input": [ 121 "input": [
@@ -183,7 +183,7 @@
183 "shipped_days": 11 183 "shipped_days": 11
184 } 184 }
185 ], 185 ],
186− "expected": 4004.65 186+ "expected": 4149.91
187 }, 187 },
188 { 188 {
189 "input": [ 189 "input": [
@@ -251,7 +251,7 @@
251 "shipped_days": 12 251 "shipped_days": 12
252 } 252 }
253 ], 253 ],
254− "expected": 3993.39 254+ "expected": 4134.77
255 }, 255 },
256 { 256 {
257 "input": [ 257 "input": [
@@ -292,7 +292,7 @@
292 "shipped_days": 12 292 "shipped_days": 12
293 } 293 }
294 ], 294 ],
295− "expected": 2050.91 295+ "expected": 2142.17
296 }, 296 },
297 { 297 {
298 "input": [ 298 "input": [
@@ -396,7 +396,7 @@
396 "shipped_days": 4 396 "shipped_days": 4
397 } 397 }
398 ], 398 ],
399− "expected": 5916.65 399+ "expected": 6153.7
400 } 400 }
401 ] 401 ]
402 } 402 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00237_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00237_2000", 2 "document_id": "exec_00237_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "455d86e6a8f1e9be0eaebb2e2ab279c0439c6768273a410e7994bc6a76dfbecf", 5+ "document_hash": "a8320d4e9d153e4398af56ca92c73f39bb1c1b9e97609e9ec096c1157cebe872",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 755 30+ "char_end": 628
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -178,7 +178,7 @@
178 "shipped_days": 10 178 "shipped_days": 10
179 } 179 }
180 ], 180 ],
181− "expected": 11415.0 181+ "expected": 11767.0
182 }, 182 },
183 { 183 {
184 "input": [ 184 "input": [
@@ -318,7 +318,7 @@
318 "shipped_days": 10 318 "shipped_days": 10
319 } 319 }
320 ], 320 ],
321− "expected": 12985.0 321+ "expected": 13403.0
322 }, 322 },
323 { 323 {
324 "input": [ 324 "input": [
@@ -467,7 +467,7 @@
467 "shipped_days": 4 467 "shipped_days": 4
468 } 468 }
469 ], 469 ],
470− "expected": 12638.0 470+ "expected": 13134.0
471 }, 471 },
472 { 472 {
473 "input": [ 473 "input": [
@@ -553,7 +553,7 @@
553 "shipped_days": 4 553 "shipped_days": 4
554 } 554 }
555 ], 555 ],
556− "expected": 5676.0 556+ "expected": 5875.0
557 }, 557 },
558 { 558 {
559 "input": [ 559 "input": [
@@ -648,7 +648,7 @@
648 "shipped_days": 13 648 "shipped_days": 13
649 } 649 }
650 ], 650 ],
651− "expected": 7335.0 651+ "expected": 7713.0
652 } 652 }
653 ] 653 ]
654 } 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00238_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00238_2000", 2 "document_id": "exec_00238_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "ffdd32b5c64c4cab201265cf785b962e0e60f2f65cf46cfd2674256d0a97fe14", 5+ "document_hash": "068a11a09e40cb07626e21d7abb095fb9c4779acc1a0db602ca10678ae79e3e0",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 749 30+ "char_end": 622
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 "shipped_days": 3 88 "shipped_days": 3
89 } 89 }
90 ], 90 ],
91− "expected": 3129.0 91+ "expected": 3164.0
92 }, 92 },
93 { 93 {
94 "input": [ 94 "input": [
@@ -210,7 +210,7 @@
210 "shipped_days": 14 210 "shipped_days": 14
211 } 211 }
212 ], 212 ],
213− "expected": 5186.0 213+ "expected": 5282.0
214 }, 214 },
215 { 215 {
216 "input": [ 216 "input": [
@@ -260,7 +260,7 @@
260 "shipped_days": 13 260 "shipped_days": 13
261 } 261 }
262 ], 262 ],
263− "expected": 1745.0 263+ "expected": 1812.0
264 }, 264 },
265 { 265 {
266 "input": [ 266 "input": [
@@ -445,7 +445,7 @@
445 "shipped_days": 1 445 "shipped_days": 1
446 } 446 }
447 ], 447 ],
448− "expected": 7412.0 448+ "expected": 7509.0
449 }, 449 },
450 { 450 {
451 "input": [ 451 "input": [
@@ -594,7 +594,7 @@
594 "shipped_days": 11 594 "shipped_days": 11
595 } 595 }
596 ], 596 ],
597− "expected": 5050.0 597+ "expected": 5126.0
598 } 598 }
599 ] 599 ]
600 } 600 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00239_2000.json +8 −8
@@ -2,13 +2,13 @@
2 "document_id": "exec_00239_2000", 2 "document_id": "exec_00239_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "1215eafbd18b1d396d2779e2c0227bfcff05e59f1f06d602fc8284f18da6bbc8", 5+ "document_hash": "015c78e1e480fb805976a26912f0308a79a0c4b2122313463b5ff6b5ee996a9b",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 4, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 14.975}", 26 "value": "{\"thr_days\": 4, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 14.975}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 758 30+ "char_end": 631
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 "shipped_days": 1 115 "shipped_days": 1
116 } 116 }
117 ], 117 ],
118− "expected": 3381.09 118+ "expected": 3407.42
119 }, 119 },
120 { 120 {
121 "input": [ 121 "input": [
@@ -269,7 +269,7 @@
269 "shipped_days": 10 269 "shipped_days": 10
270 } 270 }
271 ], 271 ],
272− "expected": 3841.81 272+ "expected": 3877.51
273 }, 273 },
274 { 274 {
275 "input": [ 275 "input": [
@@ -400,7 +400,7 @@
400 "shipped_days": 10 400 "shipped_days": 10
401 } 401 }
402 ], 402 ],
403− "expected": 8760.99 403+ "expected": 8961.66
404 }, 404 },
405 { 405 {
406 "input": [ 406 "input": [
@@ -540,7 +540,7 @@
540 "shipped_days": 6 540 "shipped_days": 6
541 } 541 }
542 ], 542 ],
543− "expected": 8834.55 543+ "expected": 9035.23
544 } 544 }
545 ] 545 ]
546 } 546 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00240_2000.json +9 −9
@@ -2,13 +2,13 @@
2 "document_id": "exec_00240_2000", 2 "document_id": "exec_00240_2000",
3 "family": "exec", 3 "family": "exec",
4 "split": "test", 4 "split": "test",
5− "document_hash": "2598ae3b4041685f7fbf0de4eb4c12a5508a3c374ae89cdf7007d3cf7e6e5924", 5+ "document_hash": "4c3927b8378a2b949e85371101d959a0adebcbe41a2d39f239aae883de382e19",
6 "questions": [ 6 "questions": [
7 { 7 {
8 "question_id": "q00_exec", 8 "question_id": "q00_exec",
9 "qtype": "exec", 9 "qtype": "exec",
10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.", 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11− "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n", 11+ "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 "answer_type": "exec", 12 "answer_type": "exec",
13 "tolerance": 0.0, 13 "tolerance": 0.0,
14 "supporting_fact_ids": [ 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 "attribute": "rules", 25 "attribute": "rules",
26 "value": "{\"thr_days\": 8, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 0.0}", 26 "value": "{\"thr_days\": 8, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 0.0}",
27 "kind": "spec", 27 "kind": "spec",
28− "sentence": "Bug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).", 28+ "sentence": "Bug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 "char_start": 20, 29 "char_start": 20,
30− "char_end": 751 30+ "char_end": 624
31 } 31 }
32 ], 32 ],
33 "hidden_tests": [ 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 "shipped_days": 6 106 "shipped_days": 6
107 } 107 }
108 ], 108 ],
109− "expected": 4847.2 109+ "expected": 4959.7
110 }, 110 },
111 { 111 {
112 "input": [ 112 "input": [
@@ -228,7 +228,7 @@
228 "shipped_days": 6 228 "shipped_days": 6
229 } 229 }
230 ], 230 ],
231− "expected": 9420.6 231+ "expected": 9646.5
232 }, 232 },
233 { 233 {
234 "input": [ 234 "input": [
@@ -278,7 +278,7 @@
278 "shipped_days": 13 278 "shipped_days": 13
279 } 279 }
280 ], 280 ],
281− "expected": 3825.6 281+ "expected": 3866.6
282 }, 282 },
283 { 283 {
284 "input": [ 284 "input": [
@@ -382,7 +382,7 @@
382 "shipped_days": 6 382 "shipped_days": 6
383 } 383 }
384 ], 384 ],
385− "expected": 7704.3 385+ "expected": 7897.0
386 }, 386 },
387 { 387 {
388 "input": [ 388 "input": [
@@ -504,7 +504,7 @@
504 "shipped_days": 8 504 "shipped_days": 8
505 } 505 }
506 ], 506 ],
507− "expected": 9420.6 507+ "expected": 9646.5
508 } 508 }
509 ] 509 ]
510 } 510 }
\ No newline at end of file
modified data/manifests/splits.json +181 −181
@@ -1,5 +1,5 @@
1 { 1 {
2− "created_utc": "2026-10-30T03:19:45.688979Z", 2+ "created_utc": "2026-10-30T03:30:41.472673Z",
3 "config": { 3 "config": {
4 "synthetic": { 4 "synthetic": {
5 "n_projects": 6, 5 "n_projects": 6,
@@ -1269,8 +1269,8 @@
1269 "document_id": "exec_00001_2000", 1269 "document_id": "exec_00001_2000",
1270 "family": "exec", 1270 "family": "exec",
1271 "split": "train", 1271 "split": "train",
1272− "document_hash": "ddfe7fcc3baca3e97144b66188f285a0e8a83018d1ca41b7f37cb80dede62dcc", 1272+ "document_hash": "72a702cc7076fb313f97772498675d83532bf6ce59f1d3cbf3bb86fa387dc0e6",
1273− "char_len": 8073, 1273+ "char_len": 7894,
1274 "n_questions": 1, 1274 "n_questions": 1,
1275 "cluster_id": "exec_00001_2000", 1275 "cluster_id": "exec_00001_2000",
1276 "seed": 1 1276 "seed": 1
@@ -1279,8 +1279,8 @@
1279 "document_id": "exec_00002_2000", 1279 "document_id": "exec_00002_2000",
1280 "family": "exec", 1280 "family": "exec",
1281 "split": "train", 1281 "split": "train",
1282− "document_hash": "7918e3cdcab2a8bfae77558b7b523f4cea70cdc52435393d98d0c7b36de29e60", 1282+ "document_hash": "7ec85eb17fda1c57c2a77a31dc79bf35b3c62974f6f1905e409880cbe869b6f5",
1283− "char_len": 8001, 1283+ "char_len": 7819,
1284 "n_questions": 1, 1284 "n_questions": 1,
1285 "cluster_id": "exec_00002_2000", 1285 "cluster_id": "exec_00002_2000",
1286 "seed": 2 1286 "seed": 2
@@ -1289,8 +1289,8 @@
1289 "document_id": "exec_00003_2000", 1289 "document_id": "exec_00003_2000",
1290 "family": "exec", 1290 "family": "exec",
1291 "split": "train", 1291 "split": "train",
1292− "document_hash": "8ab845867bddfdcb5cf63b1319a6f29794cc9dddea45cd92bca3bc94fe023da7", 1292+ "document_hash": "f7d729b32469d142625681408a1dd1aa3d1d99a7c07adb645c9f40d7550e8695",
1293− "char_len": 7972, 1293+ "char_len": 7793,
1294 "n_questions": 1, 1294 "n_questions": 1,
1295 "cluster_id": "exec_00003_2000", 1295 "cluster_id": "exec_00003_2000",
1296 "seed": 3 1296 "seed": 3
@@ -1299,8 +1299,8 @@
1299 "document_id": "exec_00004_2000", 1299 "document_id": "exec_00004_2000",
1300 "family": "exec", 1300 "family": "exec",
1301 "split": "train", 1301 "split": "train",
1302− "document_hash": "08c5cdfaa7a0742a094ee4726d97a32a89395c354270406bbcf78eff4fdbe8b2", 1302+ "document_hash": "aeca575de77be2cf25f1c9f8e053485ad918ef4b3468680a4ef6bfa3b8cb781b",
1303− "char_len": 8037, 1303+ "char_len": 7855,
1304 "n_questions": 1, 1304 "n_questions": 1,
1305 "cluster_id": "exec_00004_2000", 1305 "cluster_id": "exec_00004_2000",
1306 "seed": 4 1306 "seed": 4
@@ -1309,8 +1309,8 @@
1309 "document_id": "exec_00005_2000", 1309 "document_id": "exec_00005_2000",
1310 "family": "exec", 1310 "family": "exec",
1311 "split": "train", 1311 "split": "train",
1312− "document_hash": "f885d791d02ed1adb010c9c27804b5c38facf8fd2a09b9b1b6f9cacb7b0acc81", 1312+ "document_hash": "c5e26772985f1897f27e3729a56bdf95e08ba4718b928a1480572409c422c83f",
1313− "char_len": 8031, 1313+ "char_len": 7852,
1314 "n_questions": 1, 1314 "n_questions": 1,
1315 "cluster_id": "exec_00005_2000", 1315 "cluster_id": "exec_00005_2000",
1316 "seed": 5 1316 "seed": 5
@@ -1319,8 +1319,8 @@
1319 "document_id": "exec_00006_2000", 1319 "document_id": "exec_00006_2000",
1320 "family": "exec", 1320 "family": "exec",
1321 "split": "train", 1321 "split": "train",
1322− "document_hash": "2a1bb6d86c77861b166e281bb1deac9a658ed4383a7ade93e9698e33861de0e5", 1322+ "document_hash": "283461ef1dfaf2225620026d081a07116ec5fa2173ffa118704c0cf1d53a2eac",
1323− "char_len": 8049, 1323+ "char_len": 7867,
1324 "n_questions": 1, 1324 "n_questions": 1,
1325 "cluster_id": "exec_00006_2000", 1325 "cluster_id": "exec_00006_2000",
1326 "seed": 6 1326 "seed": 6
@@ -1329,8 +1329,8 @@
1329 "document_id": "exec_00007_2000", 1329 "document_id": "exec_00007_2000",
1330 "family": "exec", 1330 "family": "exec",
1331 "split": "train", 1331 "split": "train",
1332− "document_hash": "fb0150ce17a4b0ade1601cf6082b82e83f67db83a57c177064777d028b33d97c", 1332+ "document_hash": "961b89accd7b0332a165c291bb3c527fd00bea183807cff965326b5f758f1b7f",
1333− "char_len": 7826, 1333+ "char_len": 7644,
1334 "n_questions": 1, 1334 "n_questions": 1,
1335 "cluster_id": "exec_00007_2000", 1335 "cluster_id": "exec_00007_2000",
1336 "seed": 7 1336 "seed": 7
@@ -1339,8 +1339,8 @@
1339 "document_id": "exec_00008_2000", 1339 "document_id": "exec_00008_2000",
1340 "family": "exec", 1340 "family": "exec",
1341 "split": "train", 1341 "split": "train",
1342− "document_hash": "04f896fa681f3ae7cbfc7efa30fd96c43be9d9b703dab76457ee8043483b3612", 1342+ "document_hash": "4ad9a1fbe7db8f523841d367742505a5c92581010a15fdabb2bb6b6d77afbaa4",
1343− "char_len": 8052, 1343+ "char_len": 7873,
1344 "n_questions": 1, 1344 "n_questions": 1,
1345 "cluster_id": "exec_00008_2000", 1345 "cluster_id": "exec_00008_2000",
1346 "seed": 8 1346 "seed": 8
@@ -1349,8 +1349,8 @@
1349 "document_id": "exec_00009_2000", 1349 "document_id": "exec_00009_2000",
1350 "family": "exec", 1350 "family": "exec",
1351 "split": "train", 1351 "split": "train",
1352− "document_hash": "db44cbab6b33239dddaa205976960167be792c1b1025bd39a4c4d6cf0f6ab7d7", 1352+ "document_hash": "7c7609e3e3780713446d7ec4cbe48dba3ad69a1bf6d96393eb7e34fe13581d81",
1353− "char_len": 8097, 1353+ "char_len": 7916,
1354 "n_questions": 1, 1354 "n_questions": 1,
1355 "cluster_id": "exec_00009_2000", 1355 "cluster_id": "exec_00009_2000",
1356 "seed": 9 1356 "seed": 9
@@ -1359,8 +1359,8 @@
1359 "document_id": "exec_00010_2000", 1359 "document_id": "exec_00010_2000",
1360 "family": "exec", 1360 "family": "exec",
1361 "split": "train", 1361 "split": "train",
1362− "document_hash": "232fe52471a905a3ae14b591781c17dfa57a2d115258bfc1f8a1fce231b705ad", 1362+ "document_hash": "2951e556f1045edd4bb9926de1dac81793260ddcd823e8c6abe38566b4cbcc07",
1363− "char_len": 8004, 1363+ "char_len": 7825,
1364 "n_questions": 1, 1364 "n_questions": 1,
1365 "cluster_id": "exec_00010_2000", 1365 "cluster_id": "exec_00010_2000",
1366 "seed": 10 1366 "seed": 10
@@ -1369,8 +1369,8 @@
1369 "document_id": "exec_00011_2000", 1369 "document_id": "exec_00011_2000",
1370 "family": "exec", 1370 "family": "exec",
1371 "split": "train", 1371 "split": "train",
1372− "document_hash": "f046c8292307adf97cae067d41f5a7bcce9d2a48bc8dd143e0f11e89a1c3a4c2", 1372+ "document_hash": "6b5b870bbcdf744c667890cf6185717b10f87e45ad7a6ccada501987d53f5e6d",
1373− "char_len": 8135, 1373+ "char_len": 7953,
1374 "n_questions": 1, 1374 "n_questions": 1,
1375 "cluster_id": "exec_00011_2000", 1375 "cluster_id": "exec_00011_2000",
1376 "seed": 11 1376 "seed": 11
@@ -1379,8 +1379,8 @@
1379 "document_id": "exec_00012_2000", 1379 "document_id": "exec_00012_2000",
1380 "family": "exec", 1380 "family": "exec",
1381 "split": "train", 1381 "split": "train",
1382− "document_hash": "2befcba5bfd5b509a9a6ec5d9334e2e384a05b6733ac02d80222302d3ca6e6ef", 1382+ "document_hash": "8991dd153dfd83b4a1aea0eb4340aabe4370ac72ef16427f5f67894ac175edde",
1383− "char_len": 7906, 1383+ "char_len": 7724,
1384 "n_questions": 1, 1384 "n_questions": 1,
1385 "cluster_id": "exec_00012_2000", 1385 "cluster_id": "exec_00012_2000",
1386 "seed": 12 1386 "seed": 12
@@ -1389,8 +1389,8 @@
1389 "document_id": "exec_00013_2000", 1389 "document_id": "exec_00013_2000",
1390 "family": "exec", 1390 "family": "exec",
1391 "split": "train", 1391 "split": "train",
1392− "document_hash": "d006207e385cff8703be29071faf5845de84d05d8a677e40dda4983739115d29", 1392+ "document_hash": "ee93ee22294a4463cb6cf9ca3d5ab7ea7fd2a991b2b55448f089c2290e51a174",
1393− "char_len": 8051, 1393+ "char_len": 7870,
1394 "n_questions": 1, 1394 "n_questions": 1,
1395 "cluster_id": "exec_00013_2000", 1395 "cluster_id": "exec_00013_2000",
1396 "seed": 13 1396 "seed": 13
@@ -1399,8 +1399,8 @@
1399 "document_id": "exec_00014_2000", 1399 "document_id": "exec_00014_2000",
1400 "family": "exec", 1400 "family": "exec",
1401 "split": "train", 1401 "split": "train",
1402− "document_hash": "e6ac9234df3fc7dcec86770e041bf75b576d1c65cf6fe32717891429f5461066", 1402+ "document_hash": "b25f9d4fb9c997b0cb6e19983b48490097f6592b8d17e66f305f5a1b3e8a8443",
1403− "char_len": 8089, 1403+ "char_len": 7910,
1404 "n_questions": 1, 1404 "n_questions": 1,
1405 "cluster_id": "exec_00014_2000", 1405 "cluster_id": "exec_00014_2000",
1406 "seed": 14 1406 "seed": 14
@@ -1409,8 +1409,8 @@
1409 "document_id": "exec_00015_2000", 1409 "document_id": "exec_00015_2000",
1410 "family": "exec", 1410 "family": "exec",
1411 "split": "train", 1411 "split": "train",
1412− "document_hash": "7bd733e903db470812271aa82f1cd9c7111ace0dcd257ace35b07e660d045be7", 1412+ "document_hash": "7334bc5b260e11b2b0cdf9db05839cac2f397ae0e44db4e8bba7559dc94036bb",
1413− "char_len": 8041, 1413+ "char_len": 7859,
1414 "n_questions": 1, 1414 "n_questions": 1,
1415 "cluster_id": "exec_00015_2000", 1415 "cluster_id": "exec_00015_2000",
1416 "seed": 15 1416 "seed": 15
@@ -1419,8 +1419,8 @@
1419 "document_id": "exec_00016_2000", 1419 "document_id": "exec_00016_2000",
1420 "family": "exec", 1420 "family": "exec",
1421 "split": "train", 1421 "split": "train",
1422− "document_hash": "b232fee620440e55facf033b892741387c97661b269dc2371313fb0f5934ff69", 1422+ "document_hash": "6a5268852caa8dc3015fd42e678e912f463ba061ac417b09de9ac6ed45e85d7f",
1423− "char_len": 8045, 1423+ "char_len": 7864,
1424 "n_questions": 1, 1424 "n_questions": 1,
1425 "cluster_id": "exec_00016_2000", 1425 "cluster_id": "exec_00016_2000",
1426 "seed": 16 1426 "seed": 16
@@ -1429,8 +1429,8 @@
1429 "document_id": "exec_00017_2000", 1429 "document_id": "exec_00017_2000",
1430 "family": "exec", 1430 "family": "exec",
1431 "split": "train", 1431 "split": "train",
1432− "document_hash": "bce532bd3702b32e6eb3620ef3ea7d860a695f22bb90b8e995836a7937c77f52", 1432+ "document_hash": "f2d06422bd98b400b7401ce4c5ae2939aeb43f51482e84cba0c0e3c0845c71a5",
1433− "char_len": 8053, 1433+ "char_len": 7872,
1434 "n_questions": 1, 1434 "n_questions": 1,
1435 "cluster_id": "exec_00017_2000", 1435 "cluster_id": "exec_00017_2000",
1436 "seed": 17 1436 "seed": 17
@@ -1439,8 +1439,8 @@
1439 "document_id": "exec_00018_2000", 1439 "document_id": "exec_00018_2000",
1440 "family": "exec", 1440 "family": "exec",
1441 "split": "train", 1441 "split": "train",
1442− "document_hash": "6f0951d7af338593059efbe431f0b7ce5bf9ae652e3dfb9eedafa7e192b80c01", 1442+ "document_hash": "f219110fda4e5d20fabbe592248da3e78f61f411dbc16b9412d9b8d692eda870",
1443− "char_len": 8169, 1443+ "char_len": 7990,
1444 "n_questions": 1, 1444 "n_questions": 1,
1445 "cluster_id": "exec_00018_2000", 1445 "cluster_id": "exec_00018_2000",
1446 "seed": 18 1446 "seed": 18
@@ -1449,8 +1449,8 @@
1449 "document_id": "exec_00019_2000", 1449 "document_id": "exec_00019_2000",
1450 "family": "exec", 1450 "family": "exec",
1451 "split": "train", 1451 "split": "train",
1452− "document_hash": "733853d52a559dfc67d0cdab02de9e21c2aa09e0b25702d2cadfb38c20ab2159", 1452+ "document_hash": "232819b55ebc60215845a14aaf01c04a94dc7902933c03a47546ec41ee16502f",
1453− "char_len": 8031, 1453+ "char_len": 7849,
1454 "n_questions": 1, 1454 "n_questions": 1,
1455 "cluster_id": "exec_00019_2000", 1455 "cluster_id": "exec_00019_2000",
1456 "seed": 19 1456 "seed": 19
@@ -1459,8 +1459,8 @@
1459 "document_id": "exec_00020_2000", 1459 "document_id": "exec_00020_2000",
1460 "family": "exec", 1460 "family": "exec",
1461 "split": "train", 1461 "split": "train",
1462− "document_hash": "1a7bfffc3347254640c8b76b9b5f0ff368a758543995fa06006e28ea43d8eb71", 1462+ "document_hash": "862136f6ccb7f77097086ceddd84945e55b7df17401a63b0ce306740969966e5",
1463− "char_len": 8080, 1463+ "char_len": 7901,
1464 "n_questions": 1, 1464 "n_questions": 1,
1465 "cluster_id": "exec_00020_2000", 1465 "cluster_id": "exec_00020_2000",
1466 "seed": 20 1466 "seed": 20
@@ -1469,8 +1469,8 @@
1469 "document_id": "exec_00101_2000", 1469 "document_id": "exec_00101_2000",
1470 "family": "exec", 1470 "family": "exec",
1471 "split": "dev", 1471 "split": "dev",
1472− "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", 1472+ "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f",
1473− "char_len": 8125, 1473+ "char_len": 7944,
1474 "n_questions": 1, 1474 "n_questions": 1,
1475 "cluster_id": "exec_00101_2000", 1475 "cluster_id": "exec_00101_2000",
1476 "seed": 101 1476 "seed": 101
@@ -1479,8 +1479,8 @@
1479 "document_id": "exec_00102_2000", 1479 "document_id": "exec_00102_2000",
1480 "family": "exec", 1480 "family": "exec",
1481 "split": "dev", 1481 "split": "dev",
1482− "document_hash": "5535ac3109616f7d46dff5417535ca232754aee2c0759e072d552e465f004d46", 1482+ "document_hash": "61b6c7bb86261a26bf000996edd5f9326f7f77ca1c37085fd999cddf1bff2396",
1483− "char_len": 8089, 1483+ "char_len": 7910,
1484 "n_questions": 1, 1484 "n_questions": 1,
1485 "cluster_id": "exec_00102_2000", 1485 "cluster_id": "exec_00102_2000",
1486 "seed": 102 1486 "seed": 102
@@ -1489,8 +1489,8 @@
1489 "document_id": "exec_00103_2000", 1489 "document_id": "exec_00103_2000",
1490 "family": "exec", 1490 "family": "exec",
1491 "split": "dev", 1491 "split": "dev",
1492− "document_hash": "a7875d1e95742c6246c4ec3862b9797a4269a2b607e47869f5014d7dc29139e0", 1492+ "document_hash": "596ef1cd6d37fdac729520a738ed129e4ffaf619d372ad869d8188a0a5cac1ef",
1493− "char_len": 8038, 1493+ "char_len": 7856,
1494 "n_questions": 1, 1494 "n_questions": 1,
1495 "cluster_id": "exec_00103_2000", 1495 "cluster_id": "exec_00103_2000",
1496 "seed": 103 1496 "seed": 103
@@ -1499,8 +1499,8 @@
1499 "document_id": "exec_00104_2000", 1499 "document_id": "exec_00104_2000",
1500 "family": "exec", 1500 "family": "exec",
1501 "split": "dev", 1501 "split": "dev",
1502− "document_hash": "bce3091895fc315bf981ac1272a7da1f00b0f64819f1ecfa2a6dd8a3d23f2207", 1502+ "document_hash": "6b4aadaf1b71672d6d8aed5a412c7aa490e3e0eb1c92e5df6db1cc9554f83483",
1503− "char_len": 8014, 1503+ "char_len": 7835,
1504 "n_questions": 1, 1504 "n_questions": 1,
1505 "cluster_id": "exec_00104_2000", 1505 "cluster_id": "exec_00104_2000",
1506 "seed": 104 1506 "seed": 104
@@ -1509,8 +1509,8 @@
1509 "document_id": "exec_00105_2000", 1509 "document_id": "exec_00105_2000",
1510 "family": "exec", 1510 "family": "exec",
1511 "split": "dev", 1511 "split": "dev",
1512− "document_hash": "fe4b8857bf1774be1123bec85a5743da5c3551d53125ddf128efde401f3990c2", 1512+ "document_hash": "a59face90fe592d3d8a9b375aafefb775631dd798a60a424f981cceae1ec2996",
1513− "char_len": 8046, 1513+ "char_len": 7867,
1514 "n_questions": 1, 1514 "n_questions": 1,
1515 "cluster_id": "exec_00105_2000", 1515 "cluster_id": "exec_00105_2000",
1516 "seed": 105 1516 "seed": 105
@@ -1519,8 +1519,8 @@
1519 "document_id": "exec_00106_2000", 1519 "document_id": "exec_00106_2000",
1520 "family": "exec", 1520 "family": "exec",
1521 "split": "dev", 1521 "split": "dev",
1522− "document_hash": "c3652bfa1632f0cc62d6512131ef519100143881225c8b4ee572c1876e7c6c7f", 1522+ "document_hash": "a78f55af374fd04eb589b193a9eca2f84cf04bcdb3fa60aaeb29fbe358c79464",
1523− "char_len": 7943, 1523+ "char_len": 7761,
1524 "n_questions": 1, 1524 "n_questions": 1,
1525 "cluster_id": "exec_00106_2000", 1525 "cluster_id": "exec_00106_2000",
1526 "seed": 106 1526 "seed": 106
@@ -1529,8 +1529,8 @@
1529 "document_id": "exec_00107_2000", 1529 "document_id": "exec_00107_2000",
1530 "family": "exec", 1530 "family": "exec",
1531 "split": "dev", 1531 "split": "dev",
1532− "document_hash": "983dbcacdfae2acf0316690a7fe50cfa2523ae41beffd9de42356f2ebd3d1e79", 1532+ "document_hash": "83ea6e4f6c5d3a6eed7a921a930819d760ffb3dac7c22761dc9b4c28084904cc",
1533− "char_len": 7930, 1533+ "char_len": 7751,
1534 "n_questions": 1, 1534 "n_questions": 1,
1535 "cluster_id": "exec_00107_2000", 1535 "cluster_id": "exec_00107_2000",
1536 "seed": 107 1536 "seed": 107
@@ -1539,8 +1539,8 @@
1539 "document_id": "exec_00108_2000", 1539 "document_id": "exec_00108_2000",
1540 "family": "exec", 1540 "family": "exec",
1541 "split": "dev", 1541 "split": "dev",
1542− "document_hash": "7ebb05d3714e237d074dfc9a69c5b83e99932f9041b5d4fe15c8fc7224ba7078", 1542+ "document_hash": "b9ccaaf72598ea6e12a10942f2697dd01ca891115b652043a2f6ce4cb68570cc",
1543− "char_len": 8038, 1543+ "char_len": 7859,
1544 "n_questions": 1, 1544 "n_questions": 1,
1545 "cluster_id": "exec_00108_2000", 1545 "cluster_id": "exec_00108_2000",
1546 "seed": 108 1546 "seed": 108
@@ -1549,8 +1549,8 @@
1549 "document_id": "exec_00109_2000", 1549 "document_id": "exec_00109_2000",
1550 "family": "exec", 1550 "family": "exec",
1551 "split": "dev", 1551 "split": "dev",
1552− "document_hash": "680dab47e67683cb351941eea647197cb2b2ea472ff116582019ace0c543d461", 1552+ "document_hash": "d53ada6a135b4827dcacb2ebec142b5000ad21961fb0cd199fc7c609424d4d5a",
1553− "char_len": 8073, 1553+ "char_len": 7894,
1554 "n_questions": 1, 1554 "n_questions": 1,
1555 "cluster_id": "exec_00109_2000", 1555 "cluster_id": "exec_00109_2000",
1556 "seed": 109 1556 "seed": 109
@@ -1559,8 +1559,8 @@
1559 "document_id": "exec_00110_2000", 1559 "document_id": "exec_00110_2000",
1560 "family": "exec", 1560 "family": "exec",
1561 "split": "dev", 1561 "split": "dev",
1562− "document_hash": "f1699da28903736ad58ec0640a277bb2eac030c80c49c22c027c273374df86fa", 1562+ "document_hash": "789f1f1a750d514137a988d21a0807cc30c8548da830f2eeef66854fc179fcf5",
1563− "char_len": 8019, 1563+ "char_len": 7840,
1564 "n_questions": 1, 1564 "n_questions": 1,
1565 "cluster_id": "exec_00110_2000", 1565 "cluster_id": "exec_00110_2000",
1566 "seed": 110 1566 "seed": 110
@@ -1569,8 +1569,8 @@
1569 "document_id": "exec_00111_2000", 1569 "document_id": "exec_00111_2000",
1570 "family": "exec", 1570 "family": "exec",
1571 "split": "dev", 1571 "split": "dev",
1572− "document_hash": "e65e9534b60b15cdaadf465396c6385868f5a0ed4be5ba04c458937c2eff43ae", 1572+ "document_hash": "9f3d9d2caa1830b0bf0b983179f4de3251ec9bb8648c710bf43f0fb6c531ef85",
1573− "char_len": 7942, 1573+ "char_len": 7763,
1574 "n_questions": 1, 1574 "n_questions": 1,
1575 "cluster_id": "exec_00111_2000", 1575 "cluster_id": "exec_00111_2000",
1576 "seed": 111 1576 "seed": 111
@@ -1579,8 +1579,8 @@
1579 "document_id": "exec_00112_2000", 1579 "document_id": "exec_00112_2000",
1580 "family": "exec", 1580 "family": "exec",
1581 "split": "dev", 1581 "split": "dev",
1582− "document_hash": "4a8db801f5e7f6b2108618514f90c37fbdbb61e902c4441c0d68548a5792da05", 1582+ "document_hash": "182cf1cbb5939b7e1ca9170096aa77a9cc3c2c4f8efb3195e8b4ad19b77a3f46",
1583− "char_len": 7996, 1583+ "char_len": 7814,
1584 "n_questions": 1, 1584 "n_questions": 1,
1585 "cluster_id": "exec_00112_2000", 1585 "cluster_id": "exec_00112_2000",
1586 "seed": 112 1586 "seed": 112
@@ -1589,8 +1589,8 @@
1589 "document_id": "exec_00113_2000", 1589 "document_id": "exec_00113_2000",
1590 "family": "exec", 1590 "family": "exec",
1591 "split": "dev", 1591 "split": "dev",
1592− "document_hash": "7535b4d324a3e28b457e40fe41d18d52e75a9ed0271b411285e522c6c8c8200e", 1592+ "document_hash": "c89720bd112bca637c8fa3b4a98f7cc41a384be468eee47f5c44607ee6fc46f1",
1593− "char_len": 8055, 1593+ "char_len": 7874,
1594 "n_questions": 1, 1594 "n_questions": 1,
1595 "cluster_id": "exec_00113_2000", 1595 "cluster_id": "exec_00113_2000",
1596 "seed": 113 1596 "seed": 113
@@ -1599,8 +1599,8 @@
1599 "document_id": "exec_00114_2000", 1599 "document_id": "exec_00114_2000",
1600 "family": "exec", 1600 "family": "exec",
1601 "split": "dev", 1601 "split": "dev",
1602− "document_hash": "e1faab8175075b8cf98d2e5e16181db3f5a8f103ed89d35efee7ba4684968699", 1602+ "document_hash": "86a377fe8f0559479ca21128972c50d14bd9e19d38d2bf49f763d402259c0aad",
1603− "char_len": 8042, 1603+ "char_len": 7863,
1604 "n_questions": 1, 1604 "n_questions": 1,
1605 "cluster_id": "exec_00114_2000", 1605 "cluster_id": "exec_00114_2000",
1606 "seed": 114 1606 "seed": 114
@@ -1609,8 +1609,8 @@
1609 "document_id": "exec_00115_2000", 1609 "document_id": "exec_00115_2000",
1610 "family": "exec", 1610 "family": "exec",
1611 "split": "dev", 1611 "split": "dev",
1612− "document_hash": "4d6a9aa137c992493ef42f6758dbb35b873a4a392b00eae7152d6e45b68a3c1b", 1612+ "document_hash": "33ee1d20ea5713048e1335d3786ba0b10b3a5998a16200f09823d0c4cf0dcbe1",
1613− "char_len": 8127, 1613+ "char_len": 7946,
1614 "n_questions": 1, 1614 "n_questions": 1,
1615 "cluster_id": "exec_00115_2000", 1615 "cluster_id": "exec_00115_2000",
1616 "seed": 115 1616 "seed": 115
@@ -1619,8 +1619,8 @@
1619 "document_id": "exec_00116_2000", 1619 "document_id": "exec_00116_2000",
1620 "family": "exec", 1620 "family": "exec",
1621 "split": "dev", 1621 "split": "dev",
1622− "document_hash": "2b6100ff4b04865ebf622b9749757f3b32215faa2e32f12d63be71b0c18fc7be", 1622+ "document_hash": "379dcf9e9ada65463c8d73e2cbb54ff33c8c46ea3154539a1d6df25bab2a0a0e",
1623− "char_len": 7941, 1623+ "char_len": 7762,
1624 "n_questions": 1, 1624 "n_questions": 1,
1625 "cluster_id": "exec_00116_2000", 1625 "cluster_id": "exec_00116_2000",
1626 "seed": 116 1626 "seed": 116
@@ -1629,8 +1629,8 @@
1629 "document_id": "exec_00117_2000", 1629 "document_id": "exec_00117_2000",
1630 "family": "exec", 1630 "family": "exec",
1631 "split": "dev", 1631 "split": "dev",
1632− "document_hash": "6e53d886a35d8e51c98908f1e40e9314353c62a19e3197603509fd2a59c6bbf3", 1632+ "document_hash": "34a1cc9614d0c2d2f9766b2968ae3fa3eb2b6cc1a372750650ae7f7359325896",
1633− "char_len": 8064, 1633+ "char_len": 7883,
1634 "n_questions": 1, 1634 "n_questions": 1,
1635 "cluster_id": "exec_00117_2000", 1635 "cluster_id": "exec_00117_2000",
1636 "seed": 117 1636 "seed": 117
@@ -1639,8 +1639,8 @@
1639 "document_id": "exec_00118_2000", 1639 "document_id": "exec_00118_2000",
1640 "family": "exec", 1640 "family": "exec",
1641 "split": "dev", 1641 "split": "dev",
1642− "document_hash": "5353584056feb451c43bd2199aa87e2f7064bbfb9485b6eb1f6a1387dddac0f6", 1642+ "document_hash": "58d2108f993d9baafb67222dc4bd97895639cb6c2410e675e3ed2c4233c8fee4",
1643− "char_len": 8050, 1643+ "char_len": 7868,
1644 "n_questions": 1, 1644 "n_questions": 1,
1645 "cluster_id": "exec_00118_2000", 1645 "cluster_id": "exec_00118_2000",
1646 "seed": 118 1646 "seed": 118
@@ -1649,8 +1649,8 @@
1649 "document_id": "exec_00119_2000", 1649 "document_id": "exec_00119_2000",
1650 "family": "exec", 1650 "family": "exec",
1651 "split": "dev", 1651 "split": "dev",
1652− "document_hash": "f62c9b5ecd91caa0cf95af49bf6d40dd2bc3885255008434f482e56b7a2a2fae", 1652+ "document_hash": "98f121e7a33ff30acfcaf7a0346bbc04cc9f508e092e316490793765a8e0ab09",
1653− "char_len": 7901, 1653+ "char_len": 7722,
1654 "n_questions": 1, 1654 "n_questions": 1,
1655 "cluster_id": "exec_00119_2000", 1655 "cluster_id": "exec_00119_2000",
1656 "seed": 119 1656 "seed": 119
@@ -1659,8 +1659,8 @@
1659 "document_id": "exec_00120_2000", 1659 "document_id": "exec_00120_2000",
1660 "family": "exec", 1660 "family": "exec",
1661 "split": "dev", 1661 "split": "dev",
1662− "document_hash": "5b1b1089a793b0953579b8ce004efe99282ffd8a6eb17b16c3ce50273c121aad", 1662+ "document_hash": "c5be33e5009123ba2c251884304b839ae6fc5163f0745c302cffbabf618b40cb",
1663− "char_len": 7992, 1663+ "char_len": 7811,
1664 "n_questions": 1, 1664 "n_questions": 1,
1665 "cluster_id": "exec_00120_2000", 1665 "cluster_id": "exec_00120_2000",
1666 "seed": 120 1666 "seed": 120
@@ -1669,8 +1669,8 @@
1669 "document_id": "exec_00121_2000", 1669 "document_id": "exec_00121_2000",
1670 "family": "exec", 1670 "family": "exec",
1671 "split": "dev", 1671 "split": "dev",
1672− "document_hash": "9064e893e331010c27db6e1229c2af7da83ed7fb3680e079fc5c1a9ee7b2d73a", 1672+ "document_hash": "1645df3901b5bd61ea8c6790e15a695b1144eef69bc075121ffd0c83e6fc13af",
1673− "char_len": 8033, 1673+ "char_len": 7854,
1674 "n_questions": 1, 1674 "n_questions": 1,
1675 "cluster_id": "exec_00121_2000", 1675 "cluster_id": "exec_00121_2000",
1676 "seed": 121 1676 "seed": 121
@@ -1679,8 +1679,8 @@
1679 "document_id": "exec_00122_2000", 1679 "document_id": "exec_00122_2000",
1680 "family": "exec", 1680 "family": "exec",
1681 "split": "dev", 1681 "split": "dev",
1682− "document_hash": "8641f3a09b033f2a216948ea197682c280cda7a9559b7102473da5f152a73eb3", 1682+ "document_hash": "bd26e436a86876d04961cc142588671db4ead753f838933bde6907da5354f70e",
1683− "char_len": 8139, 1683+ "char_len": 7960,
1684 "n_questions": 1, 1684 "n_questions": 1,
1685 "cluster_id": "exec_00122_2000", 1685 "cluster_id": "exec_00122_2000",
1686 "seed": 122 1686 "seed": 122
@@ -1689,8 +1689,8 @@
1689 "document_id": "exec_00123_2000", 1689 "document_id": "exec_00123_2000",
1690 "family": "exec", 1690 "family": "exec",
1691 "split": "dev", 1691 "split": "dev",
1692− "document_hash": "3a8a85801b1200bb0a18063e20eb67097e3e9c34d009894a5a61963f7145608b", 1692+ "document_hash": "6c868c5916de1b984d70dc327ee458814726f8ea6a1fad99f8c2fb35e7e77b7f",
1693− "char_len": 8069, 1693+ "char_len": 7890,
1694 "n_questions": 1, 1694 "n_questions": 1,
1695 "cluster_id": "exec_00123_2000", 1695 "cluster_id": "exec_00123_2000",
1696 "seed": 123 1696 "seed": 123
@@ -1699,8 +1699,8 @@
1699 "document_id": "exec_00124_2000", 1699 "document_id": "exec_00124_2000",
1700 "family": "exec", 1700 "family": "exec",
1701 "split": "dev", 1701 "split": "dev",
1702− "document_hash": "adfa94aa242540ae900f58c2db73dace6e0bb6ec243a71137685b2d527e54fdb", 1702+ "document_hash": "a285cd3f7f5bbfcaa508697bf261341dc847483a97e354b95faf09bc23bb626f",
1703− "char_len": 7919, 1703+ "char_len": 7740,
1704 "n_questions": 1, 1704 "n_questions": 1,
1705 "cluster_id": "exec_00124_2000", 1705 "cluster_id": "exec_00124_2000",
1706 "seed": 124 1706 "seed": 124
@@ -1709,8 +1709,8 @@
1709 "document_id": "exec_00125_2000", 1709 "document_id": "exec_00125_2000",
1710 "family": "exec", 1710 "family": "exec",
1711 "split": "dev", 1711 "split": "dev",
1712− "document_hash": "144a4af055228fc4045ad15b1ca9ba8229c8e5563cc4c4cf9ad94203a0072dd9", 1712+ "document_hash": "386fcaff7daca426267d33af6f30657f21e4863b66d5a71d38d0b78a334eb069",
1713− "char_len": 7915, 1713+ "char_len": 7733,
1714 "n_questions": 1, 1714 "n_questions": 1,
1715 "cluster_id": "exec_00125_2000", 1715 "cluster_id": "exec_00125_2000",
1716 "seed": 125 1716 "seed": 125
@@ -1719,8 +1719,8 @@
1719 "document_id": "exec_00126_2000", 1719 "document_id": "exec_00126_2000",
1720 "family": "exec", 1720 "family": "exec",
1721 "split": "dev", 1721 "split": "dev",
1722− "document_hash": "a6bed1ca1c6bee001ded092712010d4dc8d3ec10d0d1f85432e8844be28b9359", 1722+ "document_hash": "0729452f4b76415c15bdd0075e98c4db16464515a675ea3d8e3611adb3c8a3e9",
1723− "char_len": 7944, 1723+ "char_len": 7763,
1724 "n_questions": 1, 1724 "n_questions": 1,
1725 "cluster_id": "exec_00126_2000", 1725 "cluster_id": "exec_00126_2000",
1726 "seed": 126 1726 "seed": 126
@@ -1729,8 +1729,8 @@
1729 "document_id": "exec_00127_2000", 1729 "document_id": "exec_00127_2000",
1730 "family": "exec", 1730 "family": "exec",
1731 "split": "dev", 1731 "split": "dev",
1732− "document_hash": "01e9b8b715cbfdd0d66c26d571044c18f9a6d9d2c5146b834291716879cf02d2", 1732+ "document_hash": "bb6e6467042a1b7809f3ea38b83ea82e5554ca36b70042023f682efd5bae752a",
1733− "char_len": 7953, 1733+ "char_len": 7774,
1734 "n_questions": 1, 1734 "n_questions": 1,
1735 "cluster_id": "exec_00127_2000", 1735 "cluster_id": "exec_00127_2000",
1736 "seed": 127 1736 "seed": 127
@@ -1739,8 +1739,8 @@
1739 "document_id": "exec_00128_2000", 1739 "document_id": "exec_00128_2000",
1740 "family": "exec", 1740 "family": "exec",
1741 "split": "dev", 1741 "split": "dev",
1742− "document_hash": "e702731f232f4b91d0513add4b301fd1ef0382eee35df22a98cef7eb727fafca", 1742+ "document_hash": "ea7fe68afa60ebc43cd0a501c243c72cd3b993bfaef6e389f12308e1967e3471",
1743− "char_len": 7945, 1743+ "char_len": 7763,
1744 "n_questions": 1, 1744 "n_questions": 1,
1745 "cluster_id": "exec_00128_2000", 1745 "cluster_id": "exec_00128_2000",
1746 "seed": 128 1746 "seed": 128
@@ -1749,8 +1749,8 @@
1749 "document_id": "exec_00129_2000", 1749 "document_id": "exec_00129_2000",
1750 "family": "exec", 1750 "family": "exec",
1751 "split": "dev", 1751 "split": "dev",
1752− "document_hash": "e508132470ec83c8745c1899d6c954b865130cad33382718fa48d35d6bea0ec8", 1752+ "document_hash": "46f3b799f18b08ee443481845e8695eff10ac2f2b23c2b177eafce3b42e2ea40",
1753− "char_len": 7973, 1753+ "char_len": 7794,
1754 "n_questions": 1, 1754 "n_questions": 1,
1755 "cluster_id": "exec_00129_2000", 1755 "cluster_id": "exec_00129_2000",
1756 "seed": 129 1756 "seed": 129
@@ -1759,8 +1759,8 @@
1759 "document_id": "exec_00130_2000", 1759 "document_id": "exec_00130_2000",
1760 "family": "exec", 1760 "family": "exec",
1761 "split": "dev", 1761 "split": "dev",
1762− "document_hash": "c65f14fbcb1d4d0049a6e641be2826817637866851032f95c4cc1943f87ff809", 1762+ "document_hash": "c415053d25119b89bd769d02be02c390efebcd68b992b946ddf043ad29041f86",
1763− "char_len": 8072, 1763+ "char_len": 7893,
1764 "n_questions": 1, 1764 "n_questions": 1,
1765 "cluster_id": "exec_00130_2000", 1765 "cluster_id": "exec_00130_2000",
1766 "seed": 130 1766 "seed": 130
@@ -1769,8 +1769,8 @@
1769 "document_id": "exec_00201_2000", 1769 "document_id": "exec_00201_2000",
1770 "family": "exec", 1770 "family": "exec",
1771 "split": "test", 1771 "split": "test",
1772− "document_hash": "876f090ca1801217ec4295964e7d9ce5b46b9fc96b04db7ca463d6cc8ed74978", 1772+ "document_hash": "065b6c18bbd16f16126924a54946ca4bf95c5863e57b14f33e4c9d88568f8549",
1773− "char_len": 8073, 1773+ "char_len": 7892,
1774 "n_questions": 1, 1774 "n_questions": 1,
1775 "cluster_id": "exec_00201_2000", 1775 "cluster_id": "exec_00201_2000",
1776 "seed": 201 1776 "seed": 201
@@ -1779,8 +1779,8 @@
1779 "document_id": "exec_00202_2000", 1779 "document_id": "exec_00202_2000",
1780 "family": "exec", 1780 "family": "exec",
1781 "split": "test", 1781 "split": "test",
1782− "document_hash": "660fc77600d7958889cef9b34f9c9a1a8c2d96ef38db054d180f225ebe3d4a42", 1782+ "document_hash": "ec13194237af7efe7765a21f1d8303ab8ac449271eb48e3395906648c770bab8",
1783− "char_len": 7844, 1783+ "char_len": 7665,
1784 "n_questions": 1, 1784 "n_questions": 1,
1785 "cluster_id": "exec_00202_2000", 1785 "cluster_id": "exec_00202_2000",
1786 "seed": 202 1786 "seed": 202
@@ -1789,8 +1789,8 @@
1789 "document_id": "exec_00203_2000", 1789 "document_id": "exec_00203_2000",
1790 "family": "exec", 1790 "family": "exec",
1791 "split": "test", 1791 "split": "test",
1792− "document_hash": "f26e049133c3f806d2045fa255ba4e6c2594f5ecbe872414b75faaf52ceb5033", 1792+ "document_hash": "39eee37c7ba5fe1b6d8fce9eef4c00e10a33d8ca049d12ae01b9521d4ac60863",
1793− "char_len": 7967, 1793+ "char_len": 7788,
1794 "n_questions": 1, 1794 "n_questions": 1,
1795 "cluster_id": "exec_00203_2000", 1795 "cluster_id": "exec_00203_2000",
1796 "seed": 203 1796 "seed": 203
@@ -1799,8 +1799,8 @@
1799 "document_id": "exec_00204_2000", 1799 "document_id": "exec_00204_2000",
1800 "family": "exec", 1800 "family": "exec",
1801 "split": "test", 1801 "split": "test",
1802− "document_hash": "7cc296695d97fdf8fb07de5772744af9c3a79a52f20ee1e3410187cac4dd94b2", 1802+ "document_hash": "0aaf1a095160aa21c4485db3b1b171e86d9d41b904ac2a2ea102cc13df19cd22",
1803− "char_len": 8100, 1803+ "char_len": 7921,
1804 "n_questions": 1, 1804 "n_questions": 1,
1805 "cluster_id": "exec_00204_2000", 1805 "cluster_id": "exec_00204_2000",
1806 "seed": 204 1806 "seed": 204
@@ -1809,8 +1809,8 @@
1809 "document_id": "exec_00205_2000", 1809 "document_id": "exec_00205_2000",
1810 "family": "exec", 1810 "family": "exec",
1811 "split": "test", 1811 "split": "test",
1812− "document_hash": "1f9b534774ca095126415c70bc29f4400d55f083ec020a658e6ee1263a76072f", 1812+ "document_hash": "c82f149ddbd77be80773d103825621fb187f06bb60d4fd570c73c2b3cd898ecb",
1813− "char_len": 8061, 1813+ "char_len": 7882,
1814 "n_questions": 1, 1814 "n_questions": 1,
1815 "cluster_id": "exec_00205_2000", 1815 "cluster_id": "exec_00205_2000",
1816 "seed": 205 1816 "seed": 205
@@ -1819,8 +1819,8 @@
1819 "document_id": "exec_00206_2000", 1819 "document_id": "exec_00206_2000",
1820 "family": "exec", 1820 "family": "exec",
1821 "split": "test", 1821 "split": "test",
1822− "document_hash": "71e5bf98eb246c53babd99cdd8ec1212e594a1af3ddd2a6da69bac721ad65ddd", 1822+ "document_hash": "e391c9d757bc03677c664222c3ff3f1bbab881d7f566c0072b184e0b90503339",
1823− "char_len": 7908, 1823+ "char_len": 7727,
1824 "n_questions": 1, 1824 "n_questions": 1,
1825 "cluster_id": "exec_00206_2000", 1825 "cluster_id": "exec_00206_2000",
1826 "seed": 206 1826 "seed": 206
@@ -1829,8 +1829,8 @@
1829 "document_id": "exec_00207_2000", 1829 "document_id": "exec_00207_2000",
1830 "family": "exec", 1830 "family": "exec",
1831 "split": "test", 1831 "split": "test",
1832− "document_hash": "2273befa734c54f0d780db1126ec3ce69ce8f9f829e57586a614786258681ba3", 1832+ "document_hash": "7d024469b5d38776a899aa870f5af7e146953a19f1adc1d91f634cda62aa4c14",
1833− "char_len": 8035, 1833+ "char_len": 7856,
1834 "n_questions": 1, 1834 "n_questions": 1,
1835 "cluster_id": "exec_00207_2000", 1835 "cluster_id": "exec_00207_2000",
1836 "seed": 207 1836 "seed": 207
@@ -1839,8 +1839,8 @@
1839 "document_id": "exec_00208_2000", 1839 "document_id": "exec_00208_2000",
1840 "family": "exec", 1840 "family": "exec",
1841 "split": "test", 1841 "split": "test",
1842− "document_hash": "e22e990c3c20d7feee2f70e993961e465ac68d3a867c3547dc0566739847c2d8", 1842+ "document_hash": "8528b3fafcf58b9daa88ffa9d7b9b25fb6e225f9bb767849744c05510a5ace37",
1843− "char_len": 8036, 1843+ "char_len": 7854,
1844 "n_questions": 1, 1844 "n_questions": 1,
1845 "cluster_id": "exec_00208_2000", 1845 "cluster_id": "exec_00208_2000",
1846 "seed": 208 1846 "seed": 208
@@ -1849,8 +1849,8 @@
1849 "document_id": "exec_00209_2000", 1849 "document_id": "exec_00209_2000",
1850 "family": "exec", 1850 "family": "exec",
1851 "split": "test", 1851 "split": "test",
1852− "document_hash": "bd895760d266f605848302775beaceb129633108ddc74dcbe83a8642eecb0ea3", 1852+ "document_hash": "27a0f4409b3611173adf72865b52c6beef014bce0282fd0498be50deeed192c9",
1853− "char_len": 8044, 1853+ "char_len": 7865,
1854 "n_questions": 1, 1854 "n_questions": 1,
1855 "cluster_id": "exec_00209_2000", 1855 "cluster_id": "exec_00209_2000",
1856 "seed": 209 1856 "seed": 209
@@ -1859,8 +1859,8 @@
1859 "document_id": "exec_00210_2000", 1859 "document_id": "exec_00210_2000",
1860 "family": "exec", 1860 "family": "exec",
1861 "split": "test", 1861 "split": "test",
1862− "document_hash": "2678fb8184c3c22ce8bfa861af1a68ef2704dd84cb88718d63c1b668adaf1f69", 1862+ "document_hash": "ffa82307f9b4986f3fc473b8b7df1e4f8f0e1c25d3b6e36cdcf4114a763427c5",
1863− "char_len": 7990, 1863+ "char_len": 7811,
1864 "n_questions": 1, 1864 "n_questions": 1,
1865 "cluster_id": "exec_00210_2000", 1865 "cluster_id": "exec_00210_2000",
1866 "seed": 210 1866 "seed": 210
@@ -1869,8 +1869,8 @@
1869 "document_id": "exec_00211_2000", 1869 "document_id": "exec_00211_2000",
1870 "family": "exec", 1870 "family": "exec",
1871 "split": "test", 1871 "split": "test",
1872− "document_hash": "5ad4fd06e4149025eddafab6fbf5c707e06c1f7e47f3cbaec839f52cd6744e38", 1872+ "document_hash": "95faa51b0302db54273e9b4469b6eae92c251463238fdad494d437435c293548",
1873− "char_len": 8091, 1873+ "char_len": 7909,
1874 "n_questions": 1, 1874 "n_questions": 1,
1875 "cluster_id": "exec_00211_2000", 1875 "cluster_id": "exec_00211_2000",
1876 "seed": 211 1876 "seed": 211
@@ -1879,8 +1879,8 @@
1879 "document_id": "exec_00212_2000", 1879 "document_id": "exec_00212_2000",
1880 "family": "exec", 1880 "family": "exec",
1881 "split": "test", 1881 "split": "test",
1882− "document_hash": "8a171bf76e4dc9c8921e9b2a4370fc5385ceb9695d62d19fc693110c84df06f0", 1882+ "document_hash": "2be5dc8496484a99fc11a00be474f4f7f139a8f491a2ebf3fff0db8e7804dd82",
1883− "char_len": 8061, 1883+ "char_len": 7882,
1884 "n_questions": 1, 1884 "n_questions": 1,
1885 "cluster_id": "exec_00212_2000", 1885 "cluster_id": "exec_00212_2000",
1886 "seed": 212 1886 "seed": 212
@@ -1889,8 +1889,8 @@
1889 "document_id": "exec_00213_2000", 1889 "document_id": "exec_00213_2000",
1890 "family": "exec", 1890 "family": "exec",
1891 "split": "test", 1891 "split": "test",
1892− "document_hash": "55fc33fe81ccfe25e3ebdcdb70ee135b990fcf94d02bfb56e0116982ffd6df6f", 1892+ "document_hash": "3c65c42333c25f1f3f90331747f08fafce263c8faa11a56d3a68bc58b2217327",
1893− "char_len": 7876, 1893+ "char_len": 7695,
1894 "n_questions": 1, 1894 "n_questions": 1,
1895 "cluster_id": "exec_00213_2000", 1895 "cluster_id": "exec_00213_2000",
1896 "seed": 213 1896 "seed": 213
@@ -1899,8 +1899,8 @@
1899 "document_id": "exec_00214_2000", 1899 "document_id": "exec_00214_2000",
1900 "family": "exec", 1900 "family": "exec",
1901 "split": "test", 1901 "split": "test",
1902− "document_hash": "0be81f6d80f5eb03ba862cbc94c8bf20b6ab1196483c947401585e7178fd458d", 1902+ "document_hash": "ac0a3ee69ec7da626616f6ffb5e443a4f26a3d7857caaa20f1d4ebfa5355d28b",
1903− "char_len": 7997, 1903+ "char_len": 7815,
1904 "n_questions": 1, 1904 "n_questions": 1,
1905 "cluster_id": "exec_00214_2000", 1905 "cluster_id": "exec_00214_2000",
1906 "seed": 214 1906 "seed": 214
@@ -1909,8 +1909,8 @@
1909 "document_id": "exec_00215_2000", 1909 "document_id": "exec_00215_2000",
1910 "family": "exec", 1910 "family": "exec",
1911 "split": "test", 1911 "split": "test",
1912− "document_hash": "37b95a047621c2a78beaff095b27647064c994bf6ede6ef6b3e91f8296b70eff", 1912+ "document_hash": "a4094c70181359b8442ead384be9497e3fb0c46a4d9266cce07ca77bd9e26e2b",
1913− "char_len": 8073, 1913+ "char_len": 7894,
1914 "n_questions": 1, 1914 "n_questions": 1,
1915 "cluster_id": "exec_00215_2000", 1915 "cluster_id": "exec_00215_2000",
1916 "seed": 215 1916 "seed": 215
@@ -1919,8 +1919,8 @@
1919 "document_id": "exec_00216_2000", 1919 "document_id": "exec_00216_2000",
1920 "family": "exec", 1920 "family": "exec",
1921 "split": "test", 1921 "split": "test",
1922− "document_hash": "1deddf301b0b2291cb9bdc4213848eceb17ece12032a2e454bcceb4ca4174397", 1922+ "document_hash": "1a218de32e6fe4bbe97927ea16330bda3b598ed66c1517824d94ff90f944e802",
1923− "char_len": 8017, 1923+ "char_len": 7838,
1924 "n_questions": 1, 1924 "n_questions": 1,
1925 "cluster_id": "exec_00216_2000", 1925 "cluster_id": "exec_00216_2000",
1926 "seed": 216 1926 "seed": 216
@@ -1929,8 +1929,8 @@
1929 "document_id": "exec_00217_2000", 1929 "document_id": "exec_00217_2000",
1930 "family": "exec", 1930 "family": "exec",
1931 "split": "test", 1931 "split": "test",
1932− "document_hash": "7f71cb12d994e6fe2646a0f7b337c441f923868ec6b21c29533122fda4d87a73", 1932+ "document_hash": "1223c7391b1da1aeae2d242c0e41de4639218cfe0c90ba9065f1cce42f0eedbd",
1933− "char_len": 7946, 1933+ "char_len": 7764,
1934 "n_questions": 1, 1934 "n_questions": 1,
1935 "cluster_id": "exec_00217_2000", 1935 "cluster_id": "exec_00217_2000",
1936 "seed": 217 1936 "seed": 217
@@ -1939,8 +1939,8 @@
1939 "document_id": "exec_00218_2000", 1939 "document_id": "exec_00218_2000",
1940 "family": "exec", 1940 "family": "exec",
1941 "split": "test", 1941 "split": "test",
1942− "document_hash": "2a77fd38142317b7e8868dfcaebb42248ddd462e9339268378a26e4b587eab3d", 1942+ "document_hash": "971d94c4e7b4ae02142ffb6eb92af9d2cc0f6e4d52e94a64c5132b510197ce8a",
1943− "char_len": 7955, 1943+ "char_len": 7776,
1944 "n_questions": 1, 1944 "n_questions": 1,
1945 "cluster_id": "exec_00218_2000", 1945 "cluster_id": "exec_00218_2000",
1946 "seed": 218 1946 "seed": 218
@@ -1949,8 +1949,8 @@
1949 "document_id": "exec_00219_2000", 1949 "document_id": "exec_00219_2000",
1950 "family": "exec", 1950 "family": "exec",
1951 "split": "test", 1951 "split": "test",
1952− "document_hash": "501d502ac1cd6920ddaeccbc4f0c925ddf4ca966cfb601ffd9d0547d97a741ed", 1952+ "document_hash": "390edd848add1a4cf011660e8ac9a6a6aa381c4508a8f68635f3217ce20979cb",
1953− "char_len": 8119, 1953+ "char_len": 7938,
1954 "n_questions": 1, 1954 "n_questions": 1,
1955 "cluster_id": "exec_00219_2000", 1955 "cluster_id": "exec_00219_2000",
1956 "seed": 219 1956 "seed": 219
@@ -1959,8 +1959,8 @@
1959 "document_id": "exec_00220_2000", 1959 "document_id": "exec_00220_2000",
1960 "family": "exec", 1960 "family": "exec",
1961 "split": "test", 1961 "split": "test",
1962− "document_hash": "a7487c2eb12039fcc44c64460641febdd19eaf0e13d8b6b8f00d018ca564a8b6", 1962+ "document_hash": "31eca9e7afc70ac5a795edc1d1529c531e0d028342802a61f4496c013e13bf45",
1963− "char_len": 7931, 1963+ "char_len": 7750,
1964 "n_questions": 1, 1964 "n_questions": 1,
1965 "cluster_id": "exec_00220_2000", 1965 "cluster_id": "exec_00220_2000",
1966 "seed": 220 1966 "seed": 220
@@ -1969,8 +1969,8 @@
1969 "document_id": "exec_00221_2000", 1969 "document_id": "exec_00221_2000",
1970 "family": "exec", 1970 "family": "exec",
1971 "split": "test", 1971 "split": "test",
1972− "document_hash": "a986eec8e76aa658fd694272657b8e201d940be0671e7769a934ffe76ca4748a", 1972+ "document_hash": "e228dfaa0d04173ee28b43513e5fc06914edf5d0c02b09bf0fc534531a583669",
1973− "char_len": 7820, 1973+ "char_len": 7638,
1974 "n_questions": 1, 1974 "n_questions": 1,
1975 "cluster_id": "exec_00221_2000", 1975 "cluster_id": "exec_00221_2000",
1976 "seed": 221 1976 "seed": 221
@@ -1979,8 +1979,8 @@
1979 "document_id": "exec_00222_2000", 1979 "document_id": "exec_00222_2000",
1980 "family": "exec", 1980 "family": "exec",
1981 "split": "test", 1981 "split": "test",
1982− "document_hash": "fa31c738b8b80170524adedf048965eb739a7ebf7de9fcd4f4192d331dcc3346", 1982+ "document_hash": "040260f25a42680fcc038386368506adaa004bc2592e75e3fbb5903b54cc19ab",
1983− "char_len": 8057, 1983+ "char_len": 7878,
1984 "n_questions": 1, 1984 "n_questions": 1,
1985 "cluster_id": "exec_00222_2000", 1985 "cluster_id": "exec_00222_2000",
1986 "seed": 222 1986 "seed": 222
@@ -1989,8 +1989,8 @@
1989 "document_id": "exec_00223_2000", 1989 "document_id": "exec_00223_2000",
1990 "family": "exec", 1990 "family": "exec",
1991 "split": "test", 1991 "split": "test",
1992− "document_hash": "9b2d718a88e7b451a6e6d433e81fa6c3862777588f40791fe888b2d7a44efa42", 1992+ "document_hash": "7eef13486249d89cf05bda367ea9dae2ce2479b2236c1917b2f067c49a981619",
1993− "char_len": 7992, 1993+ "char_len": 7813,
1994 "n_questions": 1, 1994 "n_questions": 1,
1995 "cluster_id": "exec_00223_2000", 1995 "cluster_id": "exec_00223_2000",
1996 "seed": 223 1996 "seed": 223
@@ -1999,8 +1999,8 @@
1999 "document_id": "exec_00224_2000", 1999 "document_id": "exec_00224_2000",
2000 "family": "exec", 2000 "family": "exec",
2001 "split": "test", 2001 "split": "test",
2002− "document_hash": "526a53c484a99dde8d83b8fc83d103060128d6331659920d7688b8836de1ec53", 2002+ "document_hash": "3e642abb57841365b827053f2437a250dcb419539389820bac778d4596c064fd",
2003− "char_len": 7921, 2003+ "char_len": 7739,
2004 "n_questions": 1, 2004 "n_questions": 1,
2005 "cluster_id": "exec_00224_2000", 2005 "cluster_id": "exec_00224_2000",
2006 "seed": 224 2006 "seed": 224
@@ -2009,8 +2009,8 @@
2009 "document_id": "exec_00225_2000", 2009 "document_id": "exec_00225_2000",
2010 "family": "exec", 2010 "family": "exec",
2011 "split": "test", 2011 "split": "test",
2012− "document_hash": "0d5d08bcef45a939605c15cb0d0b5b1ac3c2304c8358dae05fa7da75db07621f", 2012+ "document_hash": "54918c16a4ae3706a87d182112fca2a39c82c77e3a60fd25a8b47b958f2a6b21",
2013− "char_len": 7998, 2013+ "char_len": 7819,
2014 "n_questions": 1, 2014 "n_questions": 1,
2015 "cluster_id": "exec_00225_2000", 2015 "cluster_id": "exec_00225_2000",
2016 "seed": 225 2016 "seed": 225
@@ -2019,8 +2019,8 @@
2019 "document_id": "exec_00226_2000", 2019 "document_id": "exec_00226_2000",
2020 "family": "exec", 2020 "family": "exec",
2021 "split": "test", 2021 "split": "test",
2022− "document_hash": "32ca9297d301e1956917680ad370eda2add0304f9742c49a2023cfa3f2a82bd2", 2022+ "document_hash": "7a73067df93b17aba1a608bce48f7a6ff16d480af8d23713ec889405f279fc7d",
2023− "char_len": 7972, 2023+ "char_len": 7793,
2024 "n_questions": 1, 2024 "n_questions": 1,
2025 "cluster_id": "exec_00226_2000", 2025 "cluster_id": "exec_00226_2000",
2026 "seed": 226 2026 "seed": 226
@@ -2029,8 +2029,8 @@
2029 "document_id": "exec_00227_2000", 2029 "document_id": "exec_00227_2000",
2030 "family": "exec", 2030 "family": "exec",
2031 "split": "test", 2031 "split": "test",
2032− "document_hash": "0297d0253d8ecb7ba91156d4221aff1d54ba5efaed7fe7664734e63d44d31bd0", 2032+ "document_hash": "6afe19e10d57be0082db797187045b004680fa22a13a56a7b8f7e2a51b40597d",
2033− "char_len": 8074, 2033+ "char_len": 7895,
2034 "n_questions": 1, 2034 "n_questions": 1,
2035 "cluster_id": "exec_00227_2000", 2035 "cluster_id": "exec_00227_2000",
2036 "seed": 227 2036 "seed": 227
@@ -2039,8 +2039,8 @@
2039 "document_id": "exec_00228_2000", 2039 "document_id": "exec_00228_2000",
2040 "family": "exec", 2040 "family": "exec",
2041 "split": "test", 2041 "split": "test",
2042− "document_hash": "7bd8eb7ac7409f89d5f44f1f5001fd0660aece03d12d92588e821c86e0d1f69b", 2042+ "document_hash": "08d5c40fa24cccbcccaac14d408c241ab602604a712b727f12f570eb56887a74",
2043− "char_len": 7994, 2043+ "char_len": 7815,
2044 "n_questions": 1, 2044 "n_questions": 1,
2045 "cluster_id": "exec_00228_2000", 2045 "cluster_id": "exec_00228_2000",
2046 "seed": 228 2046 "seed": 228
@@ -2049,8 +2049,8 @@
2049 "document_id": "exec_00229_2000", 2049 "document_id": "exec_00229_2000",
2050 "family": "exec", 2050 "family": "exec",
2051 "split": "test", 2051 "split": "test",
2052− "document_hash": "488c5efd60f8e64b88269226071b8befe058a5e2442b99934157b38f5fc41a19", 2052+ "document_hash": "b6a34443a51bb95703036fc2467d0c50ad248f99f3c11c0a35d6bc7fcf0ad481",
2053− "char_len": 7905, 2053+ "char_len": 7723,
2054 "n_questions": 1, 2054 "n_questions": 1,
2055 "cluster_id": "exec_00229_2000", 2055 "cluster_id": "exec_00229_2000",
2056 "seed": 229 2056 "seed": 229
@@ -2059,8 +2059,8 @@
2059 "document_id": "exec_00230_2000", 2059 "document_id": "exec_00230_2000",
2060 "family": "exec", 2060 "family": "exec",
2061 "split": "test", 2061 "split": "test",
2062− "document_hash": "2e09e5c09991cb1b7c5c288b9a49bb058042a17dc7fd32cde98e2b705477598c", 2062+ "document_hash": "cbb9dba4a75d2570390205536d928042eae0341c49e7dcc11a2cea988c175ba9",
2063− "char_len": 8012, 2063+ "char_len": 7830,
2064 "n_questions": 1, 2064 "n_questions": 1,
2065 "cluster_id": "exec_00230_2000", 2065 "cluster_id": "exec_00230_2000",
2066 "seed": 230 2066 "seed": 230
@@ -2069,8 +2069,8 @@
2069 "document_id": "exec_00231_2000", 2069 "document_id": "exec_00231_2000",
2070 "family": "exec", 2070 "family": "exec",
2071 "split": "test", 2071 "split": "test",
2072− "document_hash": "9071b939e284c02834a84c70b2af1783b2a26d6217aa2184b3224f0c27400eaf", 2072+ "document_hash": "b599ec4f6ea744baef17b36d05ab828f105675eeebf81a780befd49f389ec9d3",
2073− "char_len": 8077, 2073+ "char_len": 7895,
2074 "n_questions": 1, 2074 "n_questions": 1,
2075 "cluster_id": "exec_00231_2000", 2075 "cluster_id": "exec_00231_2000",
2076 "seed": 231 2076 "seed": 231
@@ -2079,8 +2079,8 @@
2079 "document_id": "exec_00232_2000", 2079 "document_id": "exec_00232_2000",
2080 "family": "exec", 2080 "family": "exec",
2081 "split": "test", 2081 "split": "test",
2082− "document_hash": "6bb55261f58b559a6304971baddf54cabc7de9843d30afe350a659e21fa02b73", 2082+ "document_hash": "94e54bf28dac5a404fdf94451d3e632c18f63dff001cc758f89f5239d27f659b",
2083− "char_len": 7990, 2083+ "char_len": 7811,
2084 "n_questions": 1, 2084 "n_questions": 1,
2085 "cluster_id": "exec_00232_2000", 2085 "cluster_id": "exec_00232_2000",
2086 "seed": 232 2086 "seed": 232
@@ -2089,8 +2089,8 @@
2089 "document_id": "exec_00233_2000", 2089 "document_id": "exec_00233_2000",
2090 "family": "exec", 2090 "family": "exec",
2091 "split": "test", 2091 "split": "test",
2092− "document_hash": "f88eb8610ee0ae9a0cc4c953e6f302bceb7e6a9f131c2e5f775a4026d048e997", 2092+ "document_hash": "c0f56e7b9b73df623955c8b6822884e4b53df031c58736213f498ed2b4d693dc",
2093− "char_len": 7909, 2093+ "char_len": 7730,
2094 "n_questions": 1, 2094 "n_questions": 1,
2095 "cluster_id": "exec_00233_2000", 2095 "cluster_id": "exec_00233_2000",
2096 "seed": 233 2096 "seed": 233
@@ -2099,8 +2099,8 @@
2099 "document_id": "exec_00234_2000", 2099 "document_id": "exec_00234_2000",
2100 "family": "exec", 2100 "family": "exec",
2101 "split": "test", 2101 "split": "test",
2102− "document_hash": "89b49129e4581b4b424c4986c040ee62a62ff6f482d53151d96cbcf944c97e02", 2102+ "document_hash": "3522724b66617b60e55fa67efc23998ff528130352e7a960ee1e052d697670a5",
2103− "char_len": 8006, 2103+ "char_len": 7827,
2104 "n_questions": 1, 2104 "n_questions": 1,
2105 "cluster_id": "exec_00234_2000", 2105 "cluster_id": "exec_00234_2000",
2106 "seed": 234 2106 "seed": 234
@@ -2109,8 +2109,8 @@
2109 "document_id": "exec_00235_2000", 2109 "document_id": "exec_00235_2000",
2110 "family": "exec", 2110 "family": "exec",
2111 "split": "test", 2111 "split": "test",
2112− "document_hash": "7870795b3e6c98bee59eb42d0e036ccc1d88fe696a6e7c94016c77fb0f3cefe4", 2112+ "document_hash": "20bfca2f1ed7277720116b37988eb2bf9331505c61f7491ff7f131ed0b2d70e1",
2113− "char_len": 8079, 2113+ "char_len": 7900,
2114 "n_questions": 1, 2114 "n_questions": 1,
2115 "cluster_id": "exec_00235_2000", 2115 "cluster_id": "exec_00235_2000",
2116 "seed": 235 2116 "seed": 235
@@ -2119,8 +2119,8 @@
2119 "document_id": "exec_00236_2000", 2119 "document_id": "exec_00236_2000",
2120 "family": "exec", 2120 "family": "exec",
2121 "split": "test", 2121 "split": "test",
2122− "document_hash": "dabe94079fad1aa1fef7653ff5583522be249aa0c33c58334ae1d261091d83f9", 2122+ "document_hash": "23053422d45849043d9eb3706daf105a581cd66b69e96220ab1285c7c92b2058",
2123− "char_len": 8048, 2123+ "char_len": 7867,
2124 "n_questions": 1, 2124 "n_questions": 1,
2125 "cluster_id": "exec_00236_2000", 2125 "cluster_id": "exec_00236_2000",
2126 "seed": 236 2126 "seed": 236
@@ -2129,8 +2129,8 @@
2129 "document_id": "exec_00237_2000", 2129 "document_id": "exec_00237_2000",
2130 "family": "exec", 2130 "family": "exec",
2131 "split": "test", 2131 "split": "test",
2132− "document_hash": "455d86e6a8f1e9be0eaebb2e2ab279c0439c6768273a410e7994bc6a76dfbecf", 2132+ "document_hash": "a8320d4e9d153e4398af56ca92c73f39bb1c1b9e97609e9ec096c1157cebe872",
2133− "char_len": 8024, 2133+ "char_len": 7842,
2134 "n_questions": 1, 2134 "n_questions": 1,
2135 "cluster_id": "exec_00237_2000", 2135 "cluster_id": "exec_00237_2000",
2136 "seed": 237 2136 "seed": 237
@@ -2139,8 +2139,8 @@
2139 "document_id": "exec_00238_2000", 2139 "document_id": "exec_00238_2000",
2140 "family": "exec", 2140 "family": "exec",
2141 "split": "test", 2141 "split": "test",
2142− "document_hash": "ffdd32b5c64c4cab201265cf785b962e0e60f2f65cf46cfd2674256d0a97fe14", 2142+ "document_hash": "068a11a09e40cb07626e21d7abb095fb9c4779acc1a0db602ca10678ae79e3e0",
2143− "char_len": 7855, 2143+ "char_len": 7676,
2144 "n_questions": 1, 2144 "n_questions": 1,
2145 "cluster_id": "exec_00238_2000", 2145 "cluster_id": "exec_00238_2000",
2146 "seed": 238 2146 "seed": 238
@@ -2149,8 +2149,8 @@
2149 "document_id": "exec_00239_2000", 2149 "document_id": "exec_00239_2000",
2150 "family": "exec", 2150 "family": "exec",
2151 "split": "test", 2151 "split": "test",
2152− "document_hash": "1215eafbd18b1d396d2779e2c0227bfcff05e59f1f06d602fc8284f18da6bbc8", 2152+ "document_hash": "015c78e1e480fb805976a26912f0308a79a0c4b2122313463b5ff6b5ee996a9b",
2153− "char_len": 7905, 2153+ "char_len": 7723,
2154 "n_questions": 1, 2154 "n_questions": 1,
2155 "cluster_id": "exec_00239_2000", 2155 "cluster_id": "exec_00239_2000",
2156 "seed": 239 2156 "seed": 239
@@ -2159,8 +2159,8 @@
2159 "document_id": "exec_00240_2000", 2159 "document_id": "exec_00240_2000",
2160 "family": "exec", 2160 "family": "exec",
2161 "split": "test", 2161 "split": "test",
2162− "document_hash": "2598ae3b4041685f7fbf0de4eb4c12a5508a3c374ae89cdf7007d3cf7e6e5924", 2162+ "document_hash": "4c3927b8378a2b949e85371101d959a0adebcbe41a2d39f239aae883de382e19",
2163− "char_len": 7976, 2163+ "char_len": 7797,
2164 "n_questions": 1, 2164 "n_questions": 1,
2165 "cluster_id": "exec_00240_2000", 2165 "cluster_id": "exec_00240_2000",
2166 "seed": 240 2166 "seed": 240
added protocol/pilot_matrix.csv +1489 −0
@@ -0,0 +1,1489 @@
1+experiment_id,condition,method,access,sender,receiver,family,document_id,split,target_tokens,n_questions,seed,expected_trials
2+E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
3+E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
4+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
5+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
6+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
7+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
8+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
9+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
10+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
11+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
12+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
13+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
14+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
15+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
16+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
17+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
18+E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
19+E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
20+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
21+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
22+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
23+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
24+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
25+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
26+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
27+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
28+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
29+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
30+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
31+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
32+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
33+E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
34+E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
35+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
36+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
37+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
38+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
39+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
40+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
41+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
42+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
43+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
44+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
45+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
46+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
47+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
48+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
49+E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
50+E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
51+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
52+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
53+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
54+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
55+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
56+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
57+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
58+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
59+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
60+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
61+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
62+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
63+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
64+E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
65+E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
66+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
67+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
68+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
69+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
70+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
71+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
72+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
73+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
74+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
75+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
76+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
77+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
78+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
79+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
80+E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
81+E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
82+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
83+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
84+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
85+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
86+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
87+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
88+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
89+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
90+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
91+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
92+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
93+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
94+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
95+E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
96+E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
97+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
98+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
99+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
100+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
101+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
102+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
103+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
104+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
105+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
106+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
107+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
108+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
109+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
110+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
111+E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
112+E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
113+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
114+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
115+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
116+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
117+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
118+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
119+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
120+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
121+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
122+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
123+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
124+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
125+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
126+E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
127+E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
128+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
129+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
130+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
131+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
132+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
133+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
134+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
135+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
136+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
137+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
138+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
139+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
140+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
141+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
142+E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
143+E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
144+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
145+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
146+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
147+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
148+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
149+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
150+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
151+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
152+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
153+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
154+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
155+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
156+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
157+E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
158+E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
159+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
160+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
161+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
162+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
163+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
164+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
165+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
166+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
167+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
168+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
169+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
170+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
171+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
172+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
173+E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
174+E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
175+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
176+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
177+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
178+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
179+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
180+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
181+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
182+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
183+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
184+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
185+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
186+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
187+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
188+E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
189+E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
190+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
191+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
192+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
193+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
194+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
195+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
196+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
197+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
198+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
199+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
200+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
201+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
202+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
203+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
204+E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
205+E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
206+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
207+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
208+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
209+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
210+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
211+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
212+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
213+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
214+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
215+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
216+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
217+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
218+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
219+E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
220+E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
221+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
222+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
223+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
224+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
225+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
226+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
227+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
228+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
229+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
230+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
231+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
232+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
233+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
234+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
235+E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
236+E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
237+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
238+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
239+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
240+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
241+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
242+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
243+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
244+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
245+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
246+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
247+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
248+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
249+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
250+E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
251+E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
252+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
253+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
254+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
255+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
256+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
257+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
258+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
259+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
260+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
261+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
262+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
263+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
264+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
265+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
266+E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
267+E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
268+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
269+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
270+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
271+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
272+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
273+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
274+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
275+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
276+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
277+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
278+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
279+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
280+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
281+E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
282+E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
283+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
284+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
285+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
286+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
287+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
288+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
289+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
290+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
291+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
292+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
293+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
294+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
295+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
296+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
297+E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
298+E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
299+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
300+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
301+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
302+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
303+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
304+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
305+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
306+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
307+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
308+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
309+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
310+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
311+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
312+E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
313+E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
314+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
315+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
316+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
317+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
318+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
319+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
320+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
321+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
322+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
323+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
324+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
325+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
326+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
327+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
328+E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
329+E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
330+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
331+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
332+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
333+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
334+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
335+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
336+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
337+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
338+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
339+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
340+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
341+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
342+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
343+E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
344+E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
345+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
346+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
347+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
348+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
349+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
350+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
351+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
352+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
353+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
354+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
355+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
356+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
357+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
358+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
359+E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
360+E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
361+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
362+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
363+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
364+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
365+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
366+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
367+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
368+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
369+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
370+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
371+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
372+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
373+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
374+E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
375+E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
376+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
377+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
378+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
379+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
380+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
381+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
382+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
383+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
384+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
385+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
386+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
387+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
388+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
389+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
390+E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
391+E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
392+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
393+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
394+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
395+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
396+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
397+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
398+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
399+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
400+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
401+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
402+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
403+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
404+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
405+E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
406+E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
407+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
408+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
409+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
410+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
411+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
412+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
413+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
414+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
415+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
416+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
417+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
418+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
419+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
420+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
421+E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
422+E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
423+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
424+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
425+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
426+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
427+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
428+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
429+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
430+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
431+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
432+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
433+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
434+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
435+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
436+E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
437+E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
438+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
439+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
440+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
441+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
442+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
443+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
444+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
445+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
446+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
447+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
448+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
449+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
450+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
451+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
452+E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
453+E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
454+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
455+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
456+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
457+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
458+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
459+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
460+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
461+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
462+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
463+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
464+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
465+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
466+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
467+E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
468+E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
469+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
470+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
471+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
472+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
473+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
474+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
475+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
476+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
477+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
478+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
479+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
480+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
481+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
482+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
483+E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
484+E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
485+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
486+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
487+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
488+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
489+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
490+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
491+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
492+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
493+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
494+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
495+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
496+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
497+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
498+E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
499+E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
500+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
501+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
502+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
503+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
504+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
505+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
506+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
507+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
508+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
509+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
510+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
511+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
512+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
513+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
514+E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
515+E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
516+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
517+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
518+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
519+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
520+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
521+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
522+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
523+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
524+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
525+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
526+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
527+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
528+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
529+E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
530+E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
531+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
532+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
533+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
534+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
535+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
536+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
537+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
538+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
539+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
540+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
541+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
542+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
543+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
544+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
545+E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
546+E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
547+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
548+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
549+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
550+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
551+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
552+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
553+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
554+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
555+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
556+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
557+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
558+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
559+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
560+E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
561+E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
562+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
563+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
564+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
565+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
566+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
567+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
568+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
569+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
570+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
571+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
572+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
573+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
574+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
575+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
576+E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
577+E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
578+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
579+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
580+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
581+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
582+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
583+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
584+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
585+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
586+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
587+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
588+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
589+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
590+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
591+E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
592+E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
593+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
594+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
595+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
596+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
597+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
598+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
599+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
600+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
601+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
602+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
603+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
604+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
605+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
606+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
607+E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
608+E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
609+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
610+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
611+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
612+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
613+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
614+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
615+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
616+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
617+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
618+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
619+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
620+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
621+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
622+E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
623+E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
624+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
625+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
626+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
627+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
628+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
629+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
630+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
631+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
632+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
633+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
634+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
635+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
636+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
637+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
638+E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
639+E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
640+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
641+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
642+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
643+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
644+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
645+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
646+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
647+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
648+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
649+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
650+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
651+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
652+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
653+E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
654+E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
655+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
656+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
657+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
658+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
659+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
660+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
661+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
662+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
663+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
664+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
665+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
666+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
667+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
668+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
669+E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
670+E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
671+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
672+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
673+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
674+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
675+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
676+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
677+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
678+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
679+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
680+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
681+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
682+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
683+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
684+E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
685+E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
686+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
687+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
688+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
689+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
690+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
691+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
692+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
693+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
694+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
695+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
696+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
697+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
698+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
699+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
700+E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
701+E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
702+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
703+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
704+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
705+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
706+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
707+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
708+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
709+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
710+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
711+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
712+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
713+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
714+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
715+E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
716+E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
717+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
718+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
719+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
720+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
721+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
722+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
723+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
724+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
725+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
726+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
727+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
728+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
729+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
730+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
731+E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
732+E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
733+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
734+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
735+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
736+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
737+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
738+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
739+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
740+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
741+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
742+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
743+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
744+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
745+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
746+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
747+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
748+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
749+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
750+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
751+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
752+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
753+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
754+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
755+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
756+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
757+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
758+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
759+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
760+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
761+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
762+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
763+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
764+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
765+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
766+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
767+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
768+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
769+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
770+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
771+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
772+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
773+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
774+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
775+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
776+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
777+E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
778+E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
779+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
780+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
781+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
782+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
783+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
784+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
785+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
786+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
787+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
788+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
789+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
790+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
791+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
792+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
793+E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
794+E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
795+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
796+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
797+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
798+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
799+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
800+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
801+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
802+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
803+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
804+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
805+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
806+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
807+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
808+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
809+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
810+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
811+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
812+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
813+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
814+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
815+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
816+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
817+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
818+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
819+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
820+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
821+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
822+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
823+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
824+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
825+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
826+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
827+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
828+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
829+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
830+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
831+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
832+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
833+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
834+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
835+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
836+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
837+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
838+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
839+E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
840+E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
841+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
842+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
843+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
844+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
845+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
846+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
847+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
848+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
849+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
850+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
851+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
852+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
853+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
854+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
855+E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
856+E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
857+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
858+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
859+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
860+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
861+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
862+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
863+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
864+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
865+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
866+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
867+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
868+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
869+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
870+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
871+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
872+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
873+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
874+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
875+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
876+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
877+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
878+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
879+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
880+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
881+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
882+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
883+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
884+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
885+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
886+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
887+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
888+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
889+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
890+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
891+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
892+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
893+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
894+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
895+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
896+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
897+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
898+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
899+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
900+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
901+E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
902+E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
903+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
904+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
905+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
906+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
907+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
908+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
909+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
910+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
911+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
912+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
913+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
914+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
915+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
916+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
917+E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
918+E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
919+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
920+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
921+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
922+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
923+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
924+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
925+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
926+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
927+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
928+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
929+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
930+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
931+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
932+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
933+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
934+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
935+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
936+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
937+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
938+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
939+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
940+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
941+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
942+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
943+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
944+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
945+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
946+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
947+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
948+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
949+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
950+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
951+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
952+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
953+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
954+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
955+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
956+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
957+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
958+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
959+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
960+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
961+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
962+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
963+E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
964+E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
965+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
966+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
967+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
968+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
969+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
970+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
971+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
972+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
973+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
974+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
975+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
976+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
977+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
978+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
979+E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
980+E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
981+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
982+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
983+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
984+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
985+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
986+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
987+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
988+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
989+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
990+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
991+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
992+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
993+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
994+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
995+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
996+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
997+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
998+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
999+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1000+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1001+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1002+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1003+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1004+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1005+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1006+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1007+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1008+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1009+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1010+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1011+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1012+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1013+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1014+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1015+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1016+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1017+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1018+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1019+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1020+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1021+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1022+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1023+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1024+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1025+E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1026+E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1027+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1028+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1029+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1030+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1031+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1032+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1033+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1034+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1035+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1036+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1037+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1038+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1039+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1040+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1041+E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1042+E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1043+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1044+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1045+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1046+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1047+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1048+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1049+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1050+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1051+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1052+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1053+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1054+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1055+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1056+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1057+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1058+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1059+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1060+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1061+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1062+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1063+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1064+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1065+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1066+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1067+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1068+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1069+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1070+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1071+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1072+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1073+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1074+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1075+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1076+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1077+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1078+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1079+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1080+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1081+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1082+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1083+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1084+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1085+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1086+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1087+E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1088+E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1089+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1090+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1091+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1092+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1093+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1094+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1095+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1096+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1097+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1098+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1099+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1100+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1101+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1102+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1103+E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1104+E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1105+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1106+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1107+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1108+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1109+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1110+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1111+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1112+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1113+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1114+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1115+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1116+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1117+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1118+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1119+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1120+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1121+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1122+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1123+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1124+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1125+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1126+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1127+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1128+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1129+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1130+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1131+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1132+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1133+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1134+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1135+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1136+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1137+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1138+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1139+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1140+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1141+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1142+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1143+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1144+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1145+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1146+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1147+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1148+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1149+E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1150+E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1151+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1152+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1153+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1154+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1155+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1156+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1157+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1158+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1159+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1160+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1161+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1162+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1163+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1164+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1165+E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1166+E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1167+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1168+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1169+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1170+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1171+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1172+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1173+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1174+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1175+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1176+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1177+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1178+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1179+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1180+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1181+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1182+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1183+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1184+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1185+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1186+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1187+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1188+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1189+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1190+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1191+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1192+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1193+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1194+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1195+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1196+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1197+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1198+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1199+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1200+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1201+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1202+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1203+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1204+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1205+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1206+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1207+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1208+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1209+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1210+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1211+E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1212+E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1213+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1214+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1215+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1216+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1217+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1218+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1219+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1220+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1221+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1222+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1223+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1224+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1225+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1226+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1227+E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1228+E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1229+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1230+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1231+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1232+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1233+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1234+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1235+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1236+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1237+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1238+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1239+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1240+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1241+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1242+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1243+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1244+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1245+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1246+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1247+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1248+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1249+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1250+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1251+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1252+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1253+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1254+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1255+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1256+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1257+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1258+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1259+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1260+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1261+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1262+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1263+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1264+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1265+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1266+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1267+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1268+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1269+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1270+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1271+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1272+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1273+E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1274+E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1275+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1276+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1277+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1278+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1279+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1280+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1281+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1282+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1283+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1284+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1285+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1286+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1287+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1288+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1289+E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1290+E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1291+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1292+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1293+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1294+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1295+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1296+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1297+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1298+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1299+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1300+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1301+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1302+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1303+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1304+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1305+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1306+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1307+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1308+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1309+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1310+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1311+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1312+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1313+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1314+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1315+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1316+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1317+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1318+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1319+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1320+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1321+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1322+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1323+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1324+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1325+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1326+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1327+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1328+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1329+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1330+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1331+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1332+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1333+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1334+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1335+E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1336+E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1337+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1338+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1339+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1340+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1341+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1342+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1343+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1344+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1345+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1346+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1347+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1348+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1349+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1350+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1351+E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1352+E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1353+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1354+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1355+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1356+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1357+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1358+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1359+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1360+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1361+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1362+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1363+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1364+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1365+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1366+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1367+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1368+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1369+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1370+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1371+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1372+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1373+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1374+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1375+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1376+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1377+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1378+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1379+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1380+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1381+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1382+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1383+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1384+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1385+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1386+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1387+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1388+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1389+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1390+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1391+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1392+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1393+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1394+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1395+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1396+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1397+E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1398+E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1399+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1400+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1401+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1402+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1403+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1404+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1405+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1406+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1407+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1408+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1409+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1410+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1411+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1412+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1413+E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1414+E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1415+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1416+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1417+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1418+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1419+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1420+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1421+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1422+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1423+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1424+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1425+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1426+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1427+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1428+E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1429+E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1430+E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1431+E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1432+E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1433+E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1434+E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1435+E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1436+E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1437+E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1438+E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1439+E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1440+E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1441+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1442+E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1443+E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1444+E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1445+E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1446+E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1447+E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1448+E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1449+E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1450+E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1451+E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1452+E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1453+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1454+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1455+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1456+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1457+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1458+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1459+E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1460+E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1461+E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1462+E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1463+E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1464+E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1465+E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1466+E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1467+E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1468+E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1469+E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1470+E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1471+E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1472+E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1473+E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1474+E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1475+E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1476+E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1477+E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1478+E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1479+E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1480+E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1481+E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1482+E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1483+E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1484+E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1485+E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1486+E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1487+E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1488+E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1489+E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
added runs/raw/E0_smoke.jsonl +114 −0
@@ -0,0 +1,114 @@
1+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4146ad264bc41966c53c6c2a", "attempt_id": "82062e437bf8", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:34.723314Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 64, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 704.172583995387, "ttft_ms": 111.37787497136742, "decode_ms": 52.97070904634893}, "total_wall_ms": 874.542957986705, "ttft_ms": 111.37787497136742, "input_usage": {"receiver_prompt_tokens": 64}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6598574080, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6221848952, "mlx_cache_bytes": 52763976, "process_rss_bytes": 6598574080}, "peak_prefill_memory": null, "peak_decode_memory": 6221848952, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
2+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "aea3215c06f1768bed89e0e4", "attempt_id": "eb2cf16a0cb8", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:35.621639Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 70, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 714.0282499603927, "ttft_ms": 16.376250074245036, "decode_ms": 20.608124905265868}, "total_wall_ms": 751.6555410111323, "ttft_ms": 16.376250074245036, "input_usage": {"receiver_prompt_tokens": 70}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6599262208, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6243951240, "mlx_cache_bytes": 117497160, "process_rss_bytes": 6599262208}, "peak_prefill_memory": null, "peak_decode_memory": 6243951240, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
3+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "19cb26d87f4f6c06f22133ab", "attempt_id": "f2ace8e95ee4", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:36.394852Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 70, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 37.55691600963473, "ttft_ms": 13.19108298048377, "decode_ms": 21.748042083345354}, "total_wall_ms": 73.1698750751093, "ttft_ms": 13.19108298048377, "input_usage": {"receiver_prompt_tokens": 70}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6599311360, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6243951240, "mlx_cache_bytes": 117497160, "process_rss_bytes": 6599311360}, "peak_prefill_memory": null, "peak_decode_memory": 6243951240, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
4+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "55040e08a970feabf0c8dfb0", "attempt_id": "d1f4af490c6e", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:36.488122Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 67, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 33.89208402950317, "ttft_ms": 13.485999894328415, "decode_ms": 13.414166984148324}, "total_wall_ms": 61.261874972842634, "ttft_ms": 13.485999894328415, "input_usage": {"receiver_prompt_tokens": 67}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6599344128, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6245671560, "mlx_cache_bytes": 162520392, "process_rss_bytes": 6599344128}, "peak_prefill_memory": null, "peak_decode_memory": 6245671560, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
5+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b5a907e579d3ba6be2a73c6e", "attempt_id": "8a37232cf18e", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:36.569588Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2045, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 268.84245907422155, "ttft_ms": 40.27962498366833, "decode_ms": 220.9290829487145}, "total_wall_ms": 533.2459580386057, "ttft_ms": 40.27962498366833, "input_usage": {"receiver_prompt_tokens": 2045}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6599606272, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7061651504, "mlx_cache_bytes": 1144192312, "process_rss_bytes": 6599606272}, "peak_prefill_memory": null, "peak_decode_memory": 7061651504, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
6+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c3b51441b577e5cf867a711f", "attempt_id": "899cf42e0e05", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:37.123667Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 660.5961669702083, "ttft_ms": 19.695666036568582, "decode_ms": 177.584792021662}, "total_wall_ms": 861.3416670123115, "ttft_ms": 19.695666036568582, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600179712, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 1950271808, "process_rss_bytes": 6600179712}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
7+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "768e16d34f259a76a877fe66", "attempt_id": "56658b192495", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:38.004317Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 270.62474994454533, "ttft_ms": 13.784875045530498, "decode_ms": 137.02083297539502}, "total_wall_ms": 424.99795800540596, "ttft_ms": 13.784875045530498, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6600196096, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 1950271808, "process_rss_bytes": 6600196096}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
8+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bf2332e5ea92995e70a698fa", "attempt_id": "b65f009d2226", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:38.450459Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2048, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 270.55058302357793, "ttft_ms": 16.825958038680255, "decode_ms": 115.82745797932148}, "total_wall_ms": 406.5157499862835, "ttft_ms": 16.825958038680255, "input_usage": {"receiver_prompt_tokens": 2048}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6600228864, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7063175232, "mlx_cache_bytes": 1950271808, "process_rss_bytes": 6600228864}, "peak_prefill_memory": null, "peak_decode_memory": 7063175232, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
9+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "680a1696a501a9abbd246fe8", "attempt_id": "7116fac29eef", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:38.879959Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 595, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 98.40858390089124, "ttft_ms": 14.796082978136837, "decode_ms": 28.082999982871115}, "total_wall_ms": 142.88441697135568, "ttft_ms": 14.796082978136837, "input_usage": {"receiver_prompt_tokens": 595}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600474624, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6566965336, "mlx_cache_bytes": 2345519424, "process_rss_bytes": 6600474624}, "peak_prefill_memory": null, "peak_decode_memory": 6566965336, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
10+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "7d73c49fb6ea7ba015627e4f", "attempt_id": "c1a192f5a6d1", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.046782Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 601, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 96.09833301510662, "ttft_ms": 21.742249955423176, "decode_ms": 187.9519580397755}, "total_wall_ms": 307.20599996857345, "ttft_ms": 21.742249955423176, "input_usage": {"receiver_prompt_tokens": 601}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600540160, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 2695874880, "process_rss_bytes": 6600540160}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C in Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "3491362e6ed6bb557ea2c25e29286d6fdbd0a0a36995d38446eddfd8450e097c"}
11+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "610ae196b469cb2b3d8c27c1", "attempt_id": "ee38ce288b68", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.376514Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 601, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 105.36962503101677, "ttft_ms": 16.225583967752755, "decode_ms": 25.455374969169497}, "total_wall_ms": 148.63470802083611, "ttft_ms": 16.225583967752755, "input_usage": {"receiver_prompt_tokens": 601}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600556544, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 2695874880, "process_rss_bytes": 6600556544}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
12+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "17e7e463c5868b352e0a43d6", "attempt_id": "b1a1be404836", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.546022Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 598, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 95.2667499659583, "ttft_ms": 23.73883395921439, "decode_ms": 22.320416988804936}, "total_wall_ms": 142.76070904452354, "ttft_ms": 23.73883395921439, "input_usage": {"receiver_prompt_tokens": 598}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600654848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6568931416, "mlx_cache_bytes": 2941831488, "process_rss_bytes": 6600654848}, "peak_prefill_memory": null, "peak_decode_memory": 6568931416, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
13+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4e1e37e59f7beee987b1ca29", "attempt_id": "ff5f09c23dec", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.712474Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 583, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 92.41145791020244, "ttft_ms": 21.56287501566112, "decode_ms": 15.641708974726498}, "total_wall_ms": 130.70283306296915, "ttft_ms": 21.56287501566112, "input_usage": {"receiver_prompt_tokens": 583}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600769536, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6559592536, "mlx_cache_bytes": 3296414016, "process_rss_bytes": 6600769536}, "peak_prefill_memory": null, "peak_decode_memory": 6559592536, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
14+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "12de52603bea6e9934f5b230", "attempt_id": "71f4c1fad7f1", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.862544Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 589, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 93.39258295949548, "ttft_ms": 23.743500001728535, "decode_ms": 20.554958027787507}, "total_wall_ms": 139.25754197407514, "ttft_ms": 23.743500001728535, "input_usage": {"receiver_prompt_tokens": 589}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600835072, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6563131480, "mlx_cache_bytes": 3639789888, "process_rss_bytes": 6600835072}, "peak_prefill_memory": null, "peak_decode_memory": 6563131480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
15+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a5db44064cdbeec9c2302287", "attempt_id": "6e66fa4e2bab", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.022475Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 589, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 92.00699999928474, "ttft_ms": 21.959374891594052, "decode_ms": 176.00433400366455}, "total_wall_ms": 291.4037499576807, "ttft_ms": 21.959374891594052, "input_usage": {"receiver_prompt_tokens": 589}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600835072, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6563131480, "mlx_cache_bytes": 3639789888, "process_rss_bytes": 6600835072}, "peak_prefill_memory": null, "peak_decode_memory": 6563131480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,068,030", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "d1643b9a5cdbde943cffa17e0bd2ea3a63bd11939d2b80e5c53fe3f1d39a8e26"}
16+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "042990c4fab2319bcaa2df57", "attempt_id": "6f2aeaacc7b7", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.333810Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 586, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 88.77624990418553, "ttft_ms": 21.344291046261787, "decode_ms": 102.45566698722541}, "total_wall_ms": 213.67987501434982, "ttft_ms": 21.344291046261787, "input_usage": {"receiver_prompt_tokens": 586}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6600851456, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6561951832, "mlx_cache_bytes": 3880831296, "process_rss_bytes": 6600851456}, "peak_prefill_memory": null, "peak_decode_memory": 6561951832, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
17+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "8c5856e58a4069eedbaf6563", "attempt_id": "c4a0944c90e1", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.568489Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 512, "n_sentences": 24}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 587, "message_tokens": 512, "message_utf8_bytes": 1358, "cache_tensor_bytes": null, "serialized_payload_bytes": 1358, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.1340419296175241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 89.60275002755225, "ttft_ms": 20.019917050376534, "decode_ms": 194.40183392725885}, "total_wall_ms": 306.6186250653118, "ttft_ms": 20.019917050376534, "input_usage": {"receiver_prompt_tokens": 587}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6600900608, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6562246744, "mlx_cache_bytes": 4122167616, "process_rss_bytes": 6600900608}, "peak_prefill_memory": null, "peak_decode_memory": 6562246744, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "9c6f1d7683c25b99905915cd50af46f8624d0e046d60064afa1f8536d2053414", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
18+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "d2d4bfa6f46817c4a6046d3b", "attempt_id": "3a3af24c7c55", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.895456Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 25}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 589, "message_tokens": 508, "message_utf8_bytes": 1477, "cache_tensor_bytes": null, "serialized_payload_bytes": 1477, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.0682919528335333, "decode_payload_ms": 0.0, "receiver_prefill_ms": 80.28387499507517, "ttft_ms": 20.092083024792373, "decode_ms": 186.3948330283165}, "total_wall_ms": 289.03933404944837, "ttft_ms": 20.092083024792373, "input_usage": {"receiver_prompt_tokens": 589}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600916992, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6563131480, "mlx_cache_bytes": 4122167616, "process_rss_bytes": 6600916992}, "peak_prefill_memory": null, "peak_decode_memory": 6563131480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0c282c4bd4288101fb9f0b7c3f7960301f9b9cae0e1b006f3d55fa33cb6c3bdc", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
19+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "46b4c199d55ef7f74812ec8b", "attempt_id": "7ae1fd080653", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:41.203354Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 499, "n_sentences": 24}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 580, "message_tokens": 499, "message_utf8_bytes": 1320, "cache_tensor_bytes": null, "serialized_payload_bytes": 1320, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.1374169262126088, "decode_payload_ms": 0.0, "receiver_prefill_ms": 96.964625059627, "ttft_ms": 15.490291989408433, "decode_ms": 136.05291699059308}, "total_wall_ms": 250.9514580015093, "ttft_ms": 15.490291989408433, "input_usage": {"receiver_prompt_tokens": 580}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6600916992, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6577401944, "mlx_cache_bytes": 4382132544, "process_rss_bytes": 6600916992}, "peak_prefill_memory": null, "peak_decode_memory": 6577401944, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "77c2c83f97318ad674e337c84e8706903022a99e330a15111b84b24346998e7c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
20+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2919933128528059f21bcef7", "attempt_id": "bac3b7a88e92", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:41.474626Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 499, "n_sentences": 17}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 577, "message_tokens": 499, "message_utf8_bytes": 1439, "cache_tensor_bytes": null, "serialized_payload_bytes": 1439, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.8985829772427678, "decode_payload_ms": 0.0, "receiver_prefill_ms": 201.23016694560647, "ttft_ms": 18.682542024180293, "decode_ms": 100.2345000160858}, "total_wall_ms": 322.1747080096975, "ttft_ms": 18.682542024180293, "input_usage": {"receiver_prompt_tokens": 577}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6600982528, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6555218008, "mlx_cache_bytes": 4697098560, "process_rss_bytes": 6600982528}, "peak_prefill_memory": null, "peak_decode_memory": 6555218008, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "8c78c2c57027b9340eda90e83f732ddd57af36c5177bd2241b5f21f5452cfb30", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
21+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c9f341728f8327673d8c20fd", "attempt_id": "75d22a7bf90c", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:52.355810Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 591, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 96.65604203473777, "ttft_ms": 21.105334046296775, "decode_ms": 193.9961250172928}, "total_wall_ms": 313.1064580520615, "ttft_ms": 21.105334046296775, "input_usage": {"receiver_prompt_tokens": 591, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6601342976, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6584578136, "mlx_cache_bytes": 5159553344, "process_rss_bytes": 6601342976}, "peak_prefill_memory": null, "peak_decode_memory": 6584578136, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
22+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3d4dd464f7010b36c96bc271", "attempt_id": "77a451d8dea8", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:52.688816Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 99.47799996007234, "ttft_ms": 24.71804094966501, "decode_ms": 19.204750075004995}, "total_wall_ms": 144.63758398778737, "ttft_ms": 24.71804094966501, "input_usage": {"receiver_prompt_tokens": 597, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601375744, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 5404920128, "process_rss_bytes": 6601375744}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
23+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0a2034eccdf34fd25393da3d", "attempt_id": "125915e60a3b", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:52.855392Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 104.98479206580669, "ttft_ms": 21.320041036233306, "decode_ms": 13.582999934442341}, "total_wall_ms": 141.07616699766368, "ttft_ms": 21.320041036233306, "input_usage": {"receiver_prompt_tokens": 597, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601375744, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 5404920128, "process_rss_bytes": 6601375744}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
24+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e8e8d23e7cdbfbbae9a8842f", "attempt_id": "1a19d08c27d6", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:53.016421Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 98.61950005870312, "ttft_ms": 26.502083986997604, "decode_ms": 22.947457968257368}, "total_wall_ms": 149.47245793882757, "ttft_ms": 26.502083986997604, "input_usage": {"receiver_prompt_tokens": 594, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601375744, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6566572120, "mlx_cache_bytes": 5561616704, "process_rss_bytes": 6601375744}, "peak_prefill_memory": null, "peak_decode_memory": 6566572120, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
25+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bfe20256c44e4c579db64576", "attempt_id": "1297d741734b", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:03.898439Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 434, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 86.80112496949732, "ttft_ms": 15.794583945535123, "decode_ms": 15.125207952223718}, "total_wall_ms": 119.22554101329297, "ttft_ms": 15.794583945535123, "input_usage": {"receiver_prompt_tokens": 434, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601621504, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6536062772, "mlx_cache_bytes": 5921099780, "process_rss_bytes": 6601621504}, "peak_prefill_memory": null, "peak_decode_memory": 6536062772, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
26+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "14528bf1cb04b7494a446b9d", "attempt_id": "2940c8ef0c3d", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:04.040109Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 440, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 80.46558406203985, "ttft_ms": 26.1877499287948, "decode_ms": 23.025874979794025}, "total_wall_ms": 130.96516602672637, "ttft_ms": 26.1877499287948, "input_usage": {"receiver_prompt_tokens": 440, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601703424, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6540175180, "mlx_cache_bytes": 6264870624, "process_rss_bytes": 6601703424}, "peak_prefill_memory": null, "peak_decode_memory": 6540175180, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
27+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ad104e7f7b140ccf40a30520", "attempt_id": "f9ea7cc2404d", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:04.197844Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 440, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 84.51795799192041, "ttft_ms": 22.2842920338735, "decode_ms": 20.577540970407426}, "total_wall_ms": 128.7586250109598, "ttft_ms": 22.2842920338735, "input_usage": {"receiver_prompt_tokens": 440, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601736192, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6540175180, "mlx_cache_bytes": 6264870624, "process_rss_bytes": 6601736192}, "peak_prefill_memory": null, "peak_decode_memory": 6540175180, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
28+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0d6fd2ea1c04214d97935766", "attempt_id": "0efbda3ef5a6", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:04.350742Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 437, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 73.29770794603974, "ttft_ms": 22.63987495098263, "decode_ms": 22.071832907386124}, "total_wall_ms": 119.35541604179889, "ttft_ms": 22.63987495098263, "input_usage": {"receiver_prompt_tokens": 437, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601736192, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6537635660, "mlx_cache_bytes": 6504208096, "process_rss_bytes": 6601736192}, "peak_prefill_memory": null, "peak_decode_memory": 6537635660, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
29+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "cc55ee727e706809191c6b97", "attempt_id": "18419046d12f", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.157932Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1045, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 158.85141701437533, "ttft_ms": 23.864249931648374, "decode_ms": 21.887833951041102}, "total_wall_ms": 206.70400001108646, "ttft_ms": 23.864249931648374, "input_usage": {"receiver_prompt_tokens": 1045, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601932800, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6818152508, "mlx_cache_bytes": 7174788832, "process_rss_bytes": 6601932800}, "peak_prefill_memory": null, "peak_decode_memory": 6818152508, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "561d3553701d834273477e7e81d6f4a2903e1fe9e09b5b4efdd73a721bde815e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
30+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "501d065cba8addb831258e6e", "attempt_id": "de2d1ad7bd00", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.386259Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1051, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 189.56966698169708, "ttft_ms": 14.294749940745533, "decode_ms": 27.563792071305215}, "total_wall_ms": 233.33387507591397, "ttft_ms": 14.294749940745533, "input_usage": {"receiver_prompt_tokens": 1051, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601998336, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6821855292, "mlx_cache_bytes": 7792318176, "process_rss_bytes": 6601998336}, "peak_prefill_memory": null, "peak_decode_memory": 6821855292, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ab0e9c7021e297c7521daf9f4fa54de8136f15dc9e1c13a37a4af71f57687c8e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
31+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f765949413aa3fedfc527088", "attempt_id": "ceca82621c8d", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.642557Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1051, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 189.3771659815684, "ttft_ms": 25.745707913301885, "decode_ms": 17.177957925014198}, "total_wall_ms": 234.096874948591, "ttft_ms": 25.745707913301885, "input_usage": {"receiver_prompt_tokens": 1051, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6602014720, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6821855292, "mlx_cache_bytes": 7792318176, "process_rss_bytes": 6602014720}, "peak_prefill_memory": null, "peak_decode_memory": 6821855292, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ab0e9c7021e297c7521daf9f4fa54de8136f15dc9e1c13a37a4af71f57687c8e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
32+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "9b842e322458a64162511f40", "attempt_id": "c420975d5b17", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.899962Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1048, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 191.9341660104692, "ttft_ms": 21.331749972887337, "decode_ms": 21.82987506967038}, "total_wall_ms": 236.95158294867724, "ttft_ms": 21.331749972887337, "input_usage": {"receiver_prompt_tokens": 1048, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6602031104, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6850740284, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602031104}, "peak_prefill_memory": null, "peak_decode_memory": 6850740284, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ab0e9c7021e297c7521daf9f4fa54de8136f15dc9e1c13a37a4af71f57687c8e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "dbb4d6f308be12c0c7abcf21", "attempt_id": "0f02e576dce2", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:16.163883Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2045, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 351.0584579780698, "ttft_ms": 25.52862500306219, "decode_ms": 203.8207920268178}, "total_wall_ms": 583.828458096832, "ttft_ms": 25.52862500306219, "input_usage": {"receiver_prompt_tokens": 2045}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6602473472, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7061651504, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602473472}, "peak_prefill_memory": null, "peak_decode_memory": 7061651504, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
34+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a05e1aeb0d35317b7f3245e7", "attempt_id": "5ae486a7ef40", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:16.769440Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 321.8972079921514, "ttft_ms": 22.83312496729195, "decode_ms": 189.30195807479322}, "total_wall_ms": 537.459957995452, "ttft_ms": 22.83312496729195, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6602522624, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602522624}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
35+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a2c9f22c371cea273080a863", "attempt_id": "9bc42d166be3", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:17.329219Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 320.60200010892004, "ttft_ms": 21.40745904762298, "decode_ms": 157.95058303046972}, "total_wall_ms": 503.5944579867646, "ttft_ms": 21.40745904762298, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6602522624, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602522624}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
36+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0f75b3b41655640fb015977f", "attempt_id": "ce4f9d61ae8a", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:17.855898Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2048, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 304.32729201857, "ttft_ms": 20.300709060393274, "decode_ms": 107.78937500435859}, "total_wall_ms": 435.70933397859335, "ttft_ms": 20.300709060393274, "input_usage": {"receiver_prompt_tokens": 2048}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6602539008, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7063175232, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602539008}, "peak_prefill_memory": null, "peak_decode_memory": 7063175232, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
37+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c5c8f77106a5a3772c8729a5", "attempt_id": "4c8b0eb2ff80", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:18.659274Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 8.98820802103728, "receiver_prefill_ms": 0.0, "query_prefill_ms": 249.4774169754237, "ttft_ms": 249.4774169754237, "decode_ms": 216.42729197628796}, "total_wall_ms": 477.1647499874234, "ttft_ms": 249.4774169754237, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6751551488, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6445818172, "mlx_cache_bytes": 9171015533, "process_rss_bytes": 6751551488}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6445818172, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
38+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "36b870572f370219b55f990f", "attempt_id": "90b0dceb3de2", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.158034Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 0.8970829658210278, "receiver_prefill_ms": 0.0, "query_prefill_ms": 42.57166699972004, "ttft_ms": 42.57166699972004, "decode_ms": 185.1583339739591}, "total_wall_ms": 230.58637499343604, "ttft_ms": 42.57166699972004, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6751666176, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6450323772, "mlx_cache_bytes": 9211369489, "process_rss_bytes": 6751666176}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450323772, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
39+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "5c39b27a1d42aae9403aeea4", "attempt_id": "39eeebfa218d", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.410787Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 0.850292039103806, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.38766698632389, "ttft_ms": 32.38766698632389, "decode_ms": 154.78104201611131}, "total_wall_ms": 190.03108399920166, "ttft_ms": 32.38766698632389, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6751666176, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6450323684, "mlx_cache_bytes": 9211369489, "process_rss_bytes": 6751666176}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450323684, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
40+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f2fea6b40fbfa3efc8b41254", "attempt_id": "6551be76c24d", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.622622Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 0.896209036000073, "receiver_prefill_ms": 0.0, "query_prefill_ms": 208.85649998672307, "ttft_ms": 208.85649998672307, "decode_ms": 107.41054196842015}, "total_wall_ms": 318.9865410095081, "ttft_ms": 208.85649998672307, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6751764480, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6446309604, "mlx_cache_bytes": 9222920209, "process_rss_bytes": 6751764480}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6446309604, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
41+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "5242938fc112225287b86b4e", "attempt_id": "0439f90f11d2", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.986065Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 11.17420895025134, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.265333062037826, "ttft_ms": 31.265333062037826, "decode_ms": 209.63187504094094}, "total_wall_ms": 253.80166701506823, "ttft_ms": 31.265333062037826, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6826115072, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6445654220, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6826115072}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6445654220, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
42+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "7233daed51e30233779143b4", "attempt_id": "bc0068e8d567", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.262668Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 9.897915995679796, "receiver_prefill_ms": 0.0, "query_prefill_ms": 34.51216605026275, "ttft_ms": 34.51216605026275, "decode_ms": 170.15541705768555}, "total_wall_ms": 216.34687494952232, "ttft_ms": 34.51216605026275, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6832308224, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6450159844, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6832308224}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450159844, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
43+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "160b26eb5f87ef0ce84beb4d", "attempt_id": "c503e8d98716", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.502861Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 9.91812499705702, "receiver_prefill_ms": 0.0, "query_prefill_ms": 33.97083294112235, "ttft_ms": 33.97083294112235, "decode_ms": 151.9315839977935}, "total_wall_ms": 197.57162500172853, "ttft_ms": 33.97083294112235, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6838501376, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6450159844, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6838501376}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450159844, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
44+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "00a9e3a43807646a1eff16b6", "attempt_id": "b124df1d7a76", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.724657Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 7.181875058449805, "receiver_prefill_ms": 0.0, "query_prefill_ms": 38.916666992008686, "ttft_ms": 38.916666992008686, "decode_ms": 104.37366692349315}, "total_wall_ms": 152.26829203311354, "ttft_ms": 38.916666992008686, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6850887680, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6446145764, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6850887680}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6446145764, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
45+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "294ffec3fe617d6816ea913d", "attempt_id": "1828d4dd5c48", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.938893Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 12.6062500057742, "receiver_prefill_ms": 0.0, "query_prefill_ms": 273.0062920600176, "ttft_ms": 273.0062920600176, "decode_ms": 218.48266595043242}, "total_wall_ms": 505.92758296988904, "ttft_ms": 273.0062920600176, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6891094016, "backend_memory": {"mlx_active_bytes": 6331681384, "mlx_peak_bytes": 6388051512, "mlx_cache_bytes": 9450675425, "process_rss_bytes": 6891094016}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6388051512, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
46+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bcbcd6fa055fead9c2aa2d75", "attempt_id": "1b94f96b3250", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:21.469067Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 1.5839169500395656, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.846250005997717, "ttft_ms": 25.846250005997717, "decode_ms": 186.94529205095023}, "total_wall_ms": 216.11754200421274, "ttft_ms": 25.846250005997717, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6891143168, "backend_memory": {"mlx_active_bytes": 6331681384, "mlx_peak_bytes": 6397167472, "mlx_cache_bytes": 9476273137, "process_rss_bytes": 6891143168}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6397167472, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
47+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ad87548daad905473e842531", "attempt_id": "507a3aee8516", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:21.707191Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 1.4031250029802322, "receiver_prefill_ms": 0.0, "query_prefill_ms": 26.463374961167574, "ttft_ms": 26.463374961167574, "decode_ms": 176.86091596260667}, "total_wall_ms": 206.55783300753683, "ttft_ms": 26.463374961167574, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6891143168, "backend_memory": {"mlx_active_bytes": 6331665000, "mlx_peak_bytes": 6397168208, "mlx_cache_bytes": 9476289521, "process_rss_bytes": 6891143168}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6397168208, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
48+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "131a6411d0b38a14e350647f", "attempt_id": "72228fa64660", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:21.935900Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 1.620083930902183, "receiver_prefill_ms": 0.0, "query_prefill_ms": 180.2529999986291, "ttft_ms": 180.2529999986291, "decode_ms": 113.61583299003541}, "total_wall_ms": 297.15695802588016, "ttft_ms": 180.2529999986291, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6891192320, "backend_memory": {"mlx_active_bytes": 6331681384, "mlx_peak_bytes": 6389402192, "mlx_cache_bytes": 9498686449, "process_rss_bytes": 6891192320}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6389402192, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
49+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "cfa6d276303301b1b0a340f3", "attempt_id": "3ce90c85cbe1", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:22.289140Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 6.075291079469025, "receiver_prefill_ms": 0.0, "query_prefill_ms": 259.889583918266, "ttft_ms": 259.889583918266, "decode_ms": 1147.461749962531}, "total_wall_ms": 1415.1050000218675, "ttft_ms": 259.889583918266, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6912704512, "backend_memory": {"mlx_active_bytes": 6294484232, "mlx_peak_bytes": 6346778920, "mlx_cache_bytes": 9564606801, "process_rss_bytes": 6912704512}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6346778920, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "\n\nunknownunknownunknown memo\n\n5\nunknown memo\n5\n\n\n memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "78d55bf7eaf8f5d5718fae1592c5390a9d4187e68a847a5372b8f5fb9e43768c"}
50+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "34a0a1b53fd1dae009867c20", "attempt_id": "0d83c3b1bfcd", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:23.727440Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 2.2449579555541277, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.74408390279859, "ttft_ms": 32.74408390279859, "decode_ms": 1060.114708961919}, "total_wall_ms": 1096.9503748929128, "ttft_ms": 32.74408390279859, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6912737280, "backend_memory": {"mlx_active_bytes": 6294549768, "mlx_peak_bytes": 6356190592, "mlx_cache_bytes": 9564541265, "process_rss_bytes": 6912737280}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6356190592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": " \nuser\n\n memo20.\nat000000000 of000000000000000000000000 of000", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "f956396894d647df89bc96127bf3812773aa370217fc6449ccdda4300087a89f"}
51+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a8b57b89f00aa8409a780a6d", "attempt_id": "65169d0f70f2", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:24.846333Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 1.5409999759867787, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.786499958485365, "ttft_ms": 25.786499958485365, "decode_ms": 55.565542075783014}, "total_wall_ms": 84.65270895976573, "ttft_ms": 25.786499958485365, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6912737280, "backend_memory": {"mlx_active_bytes": 6294566152, "mlx_peak_bytes": 6356190592, "mlx_cache_bytes": 9564524881, "process_rss_bytes": 6912737280}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6356190592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "\n\n", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "invalid_output", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "75a11da44c802486bc6f65640aa48a730f0f684c5c07a42ba3cd1735eb3fb070"}
52+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "01f04fced02030ea8b7fb96c", "attempt_id": "91a614851ec3", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:24.952208Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 1.630165963433683, "receiver_prefill_ms": 0.0, "query_prefill_ms": 180.5562499212101, "ttft_ms": 180.5562499212101, "decode_ms": 1070.0649589998648}, "total_wall_ms": 1254.2412499897182, "ttft_ms": 180.5562499212101, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6912786432, "backend_memory": {"mlx_active_bytes": 6294500616, "mlx_peak_bytes": 6348424592, "mlx_cache_bytes": 9564590417, "process_rss_bytes": 6912786432}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6348424592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "user\n\nsystem 0 0 000000000000000000000000000000000000000", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "d3da647a46cbd3eb57b7ba22bfc90fff80e5d0dfb8efe06d0ea15bf1b7249bde"}
53+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3866a433f650f0db81383698", "attempt_id": "94d40d88413a", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.301231Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 1.023292075842619, "receiver_prefill_ms": 0.0, "query_prefill_ms": 35.569333005696535, "ttft_ms": 35.569333005696535, "decode_ms": 199.1004589945078}, "total_wall_ms": 237.6932919723913, "ttft_ms": 35.569333005696535, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6914555904, "backend_memory": {"mlx_active_bytes": 6287777800, "mlx_peak_bytes": 6324363468, "mlx_cache_bytes": 9575183873, "process_rss_bytes": 6914555904}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6324363468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "MP-89X-6320", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "e07259f8346a7570e5b980e01a868e6af8ede517e6d7f6a2958e2f3b8da08023"}
54+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c5a46a2a80d968eb5b9cfd03", "attempt_id": "d3ca7d571fc6", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.559533Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 0.7745410548523068, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.40566700231284, "ttft_ms": 27.40566700231284, "decode_ms": 165.24295799899846}, "total_wall_ms": 195.35333395469934, "ttft_ms": 27.40566700231284, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6914588672, "backend_memory": {"mlx_active_bytes": 6287777800, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9575183873, "process_rss_bytes": 6914588672}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
55+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "9e9adefa1df3c8162ef90094", "attempt_id": "d4d46f7011e0", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.777798Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 0.9067920036613941, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.181916986592114, "ttft_ms": 27.181916986592114, "decode_ms": 32.393874949775636}, "total_wall_ms": 62.11195804644376, "ttft_ms": 27.181916986592114, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6914588672, "backend_memory": {"mlx_active_bytes": 6287745032, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9575216641, "process_rss_bytes": 6914588672}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not provided", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "d83b9ff3c07a8390e2ea0a2af45fba37595da25f31b85ba625363ee98c0c796f"}
56+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a4e24a1d6a957db1e957f87c", "attempt_id": "642d96719fa3", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.863105Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 1.1591659858822823, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.929875068366528, "ttft_ms": 27.929875068366528, "decode_ms": 76.25045790337026}, "total_wall_ms": 106.98124999180436, "ttft_ms": 27.929875068366528, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6914588672, "backend_memory": {"mlx_active_bytes": 6287155208, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9575806465, "process_rss_bytes": 6914588672}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Milos Ferrante", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "14ae3adaefbe5d307a9585834a37951eb0794dc1bfacd0739a580886526143d2"}
57+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c5299d0a9079f76c811be8e1", "attempt_id": "a91029007c95", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:27.021383Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.7211669581010938, "receiver_prefill_ms": 0.0, "query_prefill_ms": 22.886250051669776, "ttft_ms": 22.886250051669776, "decode_ms": 941.9688330963254}, "total_wall_ms": 967.6620420068502, "ttft_ms": 22.886250051669776, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934003712, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6324363468, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934003712}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6324363468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant,", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "a7f3f386f2ebe03f3097faba3f505d405d26b2f9abd5b6faacedc39b50901a2d"}
58+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "92833db31d27eea8ff70489c", "attempt_id": "7e0b4656c29b", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:28.012306Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.9810830233618617, "receiver_prefill_ms": 0.0, "query_prefill_ms": 28.88324996456504, "ttft_ms": 28.88324996456504, "decode_ms": 929.5173750724643}, "total_wall_ms": 961.1695419298485, "ttft_ms": 28.88324996456504, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934020096, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934020096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "426a0f227def734f39a983ae88dcd05e8dcbe992627914324fb637eb8cb63e6b"}
59+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2cbc1b12a43668b7e57b864c", "attempt_id": "0d1f38b0b06c", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:28.997109Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.652667018584907, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.082583030685782, "ttft_ms": 29.082583030685782, "decode_ms": 957.0584170287475}, "total_wall_ms": 988.5560829425231, "ttft_ms": 29.082583030685782, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934020096, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934020096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "The Project codes are the name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "79576cc33fa68a3208636b2d8810c8c0d1f2117032776dc346a524e04fb85a25"}
60+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "169b50740e3460318aa49fa9", "attempt_id": "e99d3d382fb0", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:30.008009Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.8759999182075262, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.01162502914667, "ttft_ms": 30.01162502914667, "decode_ms": 949.0018750075251}, "total_wall_ms": 981.6403749864548, "ttft_ms": 30.01162502914667, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934020096, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934020096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "I will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\n", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "ec85516726e09bae04746f5b6be22f47099830d26c5193e069713a8e4b00dca5"}
61+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "93f6f7676d8794e607f42e04", "attempt_id": "da4d2710c0b7", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:31.010413Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 35, "compress_ms": 563.8788749929518}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 572.4881670903414, "doc_prefill_ms": 308.5541669279337, "compress_ms": 565.5872919596732, "serialize_ms": 6.614041980355978, "load_ms": 0.8810000726953149, "receiver_prefill_ms": 0.0, "query_prefill_ms": 20.365500007756054, "ttft_ms": 20.365500007756054, "decode_ms": 119.11770806182176}, "total_wall_ms": 714.6071669412777, "ttft_ms": 20.365500007756054, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": 6936444928, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6322151628, "mlx_cache_bytes": 9625236993, "process_rss_bytes": 6936444928}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6322151628, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-1482", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "f5e9a6d78321ae31e0a3bdeeaf8c39b796c1fc12a0a8896955e8c411f428367c"}
62+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2f2d1a37db90019896731035", "attempt_id": "be106ee31fdd", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:31.746833Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 41, "compress_ms": 55.423791985958815}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521686, "side_information_bytes": 15958, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.9762079808861, "doc_prefill_ms": 308.5541669279337, "compress_ms": 57.0506660733372, "serialize_ms": 6.670959061011672, "load_ms": 0.9055420523509383, "receiver_prefill_ms": 0.0, "query_prefill_ms": 28.734207968227565, "ttft_ms": 28.734207968227565, "decode_ms": 179.1473749326542}, "total_wall_ms": 274.67562502715737, "ttft_ms": 28.734207968227565, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6936526848, "backend_memory": {"mlx_active_bytes": 6287777800, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9648109057, "process_rss_bytes": 6936526848}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2133e2d8fb6fbe8bb2bc071e98a75ef68a33983b8f86299196b1d6fddcd4bf21", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
63+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3c7f1c745d25a93cc74dfa5b", "attempt_id": "ea7dbd066ab8", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.046152Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 41, "compress_ms": 60.98887498956174}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 69.41050000023097, "doc_prefill_ms": 308.5541669279337, "compress_ms": 62.77983298059553, "serialize_ms": 6.343499990180135, "load_ms": 0.7366250501945615, "receiver_prefill_ms": 0.0, "query_prefill_ms": 28.80037494469434, "ttft_ms": 28.80037494469434, "decode_ms": 144.29624995682389}, "total_wall_ms": 245.3072089701891, "ttft_ms": 28.80037494469434, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6936559616, "backend_memory": {"mlx_active_bytes": 6287745032, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9650468353, "process_rss_bytes": 6936559616}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
64+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c8f2e4a96d7379d7d34702d1", "attempt_id": "e9277718bcd6", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.313036Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 38, "compress_ms": 175.28775008395314}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521686, "side_information_bytes": 15958, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 182.38962499890476, "doc_prefill_ms": 308.5541669279337, "compress_ms": 176.8307499587536, "serialize_ms": 5.288459011353552, "load_ms": 0.5769999697804451, "receiver_prefill_ms": 0.0, "query_prefill_ms": 26.420791051350534, "ttft_ms": 26.420791051350534, "decode_ms": 105.64537497702986}, "total_wall_ms": 316.87600002624094, "ttft_ms": 26.420791051350534, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6936657920, "backend_memory": {"mlx_active_bytes": 6287745032, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9655367169, "process_rss_bytes": 6936657920}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2133e2d8fb6fbe8bb2bc071e98a75ef68a33983b8f86299196b1d6fddcd4bf21", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
65+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a0f56d97be7daffa8cef249e", "attempt_id": "3e0f81469768", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.713987Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 1.1760419001802802, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.72891598660499, "ttft_ms": 31.72891598660499, "decode_ms": 19.485582946799695}, "total_wall_ms": 54.12295798305422, "ttft_ms": 31.72891598660499, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936887296, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6322151628, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936887296}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6322151628, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
66+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "490a5844ac7e4eb254afa577", "attempt_id": "24d3c5e9fd52", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.789602Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 0.7209168979898095, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.789833045564592, "ttft_ms": 25.789833045564592, "decode_ms": 24.353457964025438}, "total_wall_ms": 52.51691699959338, "ttft_ms": 25.789833045564592, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936920064, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936920064}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
67+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f43fe1d7913c24ca7775f970", "attempt_id": "a9cecb8c4957", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.863379Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 0.816666055470705, "receiver_prefill_ms": 0.0, "query_prefill_ms": 19.551209057681262, "ttft_ms": 19.551209057681262, "decode_ms": 23.046249989420176}, "total_wall_ms": 45.09937507100403, "ttft_ms": 19.551209057681262, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936920064, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936920064}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
68+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "23953c62f9522d607c97ebc1", "attempt_id": "df96117d0a60", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.930619Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 1.2563749914988875, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.40870802477002, "ttft_ms": 25.40870802477002, "decode_ms": 13.515500002540648}, "total_wall_ms": 41.98029194958508, "ttft_ms": 25.40870802477002, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936920064, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936920064}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
69+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e4c6ac1cbe2528e118fc5703", "attempt_id": "a857f215ff82", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.994458Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 77.96645897906274, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (35,966) and (1,16,35,942) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
70+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e7b9ff431123b343d517b9dc", "attempt_id": "900e4aabd23d", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.096831Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 66.20066694449633, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (41,972) and (1,16,41,948) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
71+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4447de7a1f609caa2fe38d9d", "attempt_id": "77c4b23465cf", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.184948Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 75.23312489502132, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (41,972) and (1,16,41,948) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
72+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "135f1a8ecad430fbb7dadf92", "attempt_id": "6f686bd52983", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.282090Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 71.89062493853271, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (38,969) and (1,16,38,945) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
73+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "56ee991b90464dd33a7c9ee5", "attempt_id": "f3c10dcea6ab", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.403424Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.8424169393256307, "receiver_prefill_ms": 0.0, "query_prefill_ms": 19.6460000006482, "ttft_ms": 19.6460000006482, "decode_ms": 52.42887500207871}, "total_wall_ms": 75.69141697604209, "ttft_ms": 19.6460000006482, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6983172096, "backend_memory": {"mlx_active_bytes": 6290394144, "mlx_peak_bytes": 6335647168, "mlx_cache_bytes": 9688339849, "process_rss_bytes": 6983172096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6335647168, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not provided", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "d83b9ff3c07a8390e2ea0a2af45fba37595da25f31b85ba625363ee98c0c796f"}
74+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2f67a67e0c52d41f79c0f979", "attempt_id": "8b2fad1d3eca", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.501685Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.884708064608276, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.582791983149946, "ttft_ms": 27.582791983149946, "decode_ms": 213.58125004917383}, "total_wall_ms": 244.85879205167294, "ttft_ms": 27.582791983149946, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6983204864, "backend_memory": {"mlx_active_bytes": 6290656288, "mlx_peak_bytes": 6352739496, "mlx_cache_bytes": 9688077705, "process_rss_bytes": 6983204864}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6352739496, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
75+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a457f580ccecc54298315139", "attempt_id": "5cf33b8c867a", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.767722Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.6323330346494913, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.149292036890984, "ttft_ms": 29.149292036890984, "decode_ms": 161.35833400767297}, "total_wall_ms": 194.04400000348687, "ttft_ms": 29.149292036890984, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6983204864, "backend_memory": {"mlx_active_bytes": 6290394144, "mlx_peak_bytes": 6352739496, "mlx_cache_bytes": 9688339849, "process_rss_bytes": 6983204864}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6352739496, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
76+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0a4e1364ef45cc38c5da4506", "attempt_id": "3b25f21b11c6", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.985086Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.4646250056102872, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.344999977387488, "ttft_ms": 30.344999977387488, "decode_ms": 122.02841695398092}, "total_wall_ms": 155.62674996908754, "ttft_ms": 30.344999977387488, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6983204864, "backend_memory": {"mlx_active_bytes": 6290361376, "mlx_peak_bytes": 6340107432, "mlx_cache_bytes": 9699874185, "process_rss_bytes": 6983204864}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6340107432, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
77+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b8d8e634e61d167ebc87181e", "attempt_id": "481c6ed42eac", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:34.241172Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 73, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011124997399747372, "decode_payload_ms": 0.0, "receiver_prefill_ms": 44.282374903559685, "ttft_ms": 19.709625048562884, "decode_ms": 20.926458993926644}, "total_wall_ms": 85.48883290495723, "ttft_ms": 19.709625048562884, "input_usage": {"receiver_prompt_tokens": 73}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6701006848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6245949864, "mlx_cache_bytes": 172412400, "process_rss_bytes": 6701006848}, "peak_prefill_memory": null, "peak_decode_memory": 6245949864, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:NameError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
78+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "826224a74ebf0be97179de8c", "attempt_id": "6aea9004d9cf", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:34.374957Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 2020, "message_tokens": 1937, "message_utf8_bytes": 8125, "cache_tensor_bytes": null, "serialized_payload_bytes": 8125, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.02150004729628563, "decode_payload_ms": 0.0, "receiver_prefill_ms": 295.2367920661345, "ttft_ms": 14.32908303104341, "decode_ms": 1010.4336669901386}, "total_wall_ms": 1323.3462499920279, "ttft_ms": 14.32908303104341, "input_usage": {"receiver_prompt_tokens": 2020}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701006848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7052050480, "mlx_cache_bytes": 1068830192, "process_rss_bytes": 6701006848}, "peak_prefill_memory": null, "peak_decode_memory": 7052050480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
79+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b723bddfa5449bee38ed407a", "attempt_id": "4a8fdefb533f", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:35.745633Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1937}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 604, "message_tokens": 516, "message_utf8_bytes": 2156, "cache_tensor_bytes": null, "serialized_payload_bytes": 2156, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.090957947075367, "decode_payload_ms": 0.0, "receiver_prefill_ms": 93.57858297880739, "ttft_ms": 23.513957974500954, "decode_ms": 911.6655830293894}, "total_wall_ms": 1030.1828749943525, "ttft_ms": 23.513957974500954, "input_usage": {"receiver_prompt_tokens": 604}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701006848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6569799768, "mlx_cache_bytes": 1469386224, "process_rss_bytes": 6701006848}, "peak_prefill_memory": null, "peak_decode_memory": 6569799768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['region'] != 'Sud' and row['units'] >= 2 and row['shipped_days'] <= ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5668e3e1f062ec6fbeef2cd13bd6309d297d591e881d4bbc87cf51b2a5cfaf9f", "output_hash": "49013de476d48dbb78217dba376506dfae969e7e8e9e11415afad7ea2f53b479"}
80+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "162117ede3591cddd228532b", "attempt_id": "e08d4687822b", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:36.823140Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 500, "n_sentences": 3}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 584, "message_tokens": 499, "message_utf8_bytes": 1474, "cache_tensor_bytes": null, "serialized_payload_bytes": 1474, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.0204579923301935, "decode_payload_ms": 0.0, "receiver_prefill_ms": 92.08316600415856, "ttft_ms": 20.726916030980647, "decode_ms": 923.1244999682531}, "total_wall_ms": 1037.3579170554876, "ttft_ms": 20.726916030980647, "input_usage": {"receiver_prompt_tokens": 584}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701056000, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6559985752, "mlx_cache_bytes": 1824361968, "process_rss_bytes": 6701056000}, "peak_prefill_memory": null, "peak_decode_memory": 6559985752, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['region'] == 'Sud' or row['units'] < 2 or row['returned']:\n continue\n ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0d0b68d8ccb32536ba23b7d25e38a4021a853137ff08f20815b4c3d2b9b45d64", "output_hash": "99475af42ec047e0ce1a7d674ae59578eb16d871e0e2d91661b164982ead65f8"}
81+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "146dc78b2613ad3e8f9ca821", "attempt_id": "6654ee0f823d", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:37.904166Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 506, "n_sentences": 22}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 590, "message_tokens": 506, "message_utf8_bytes": 2318, "cache_tensor_bytes": null, "serialized_payload_bytes": 2318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.4914999483153224, "decode_payload_ms": 0.0, "receiver_prefill_ms": 98.94125000573695, "ttft_ms": 16.27620798535645, "decode_ms": 934.4471250660717}, "total_wall_ms": 1052.851833985187, "ttft_ms": 16.27620798535645, "input_usage": {"receiver_prompt_tokens": 590}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701056000, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6564016216, "mlx_cache_bytes": 2183368176, "process_rss_bytes": 6701056000}, "peak_prefill_memory": null, "peak_decode_memory": 6564016216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for r in rows:\n if r['region'] == 'Sud' or r['units'] < 2 or r['shipped_days'] > ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "32a2539d2b6ee4c06cee34a3497e76891f4e908fa56a9f3359ae6958be2a1961", "output_hash": "740d0b6993023215fc488fca5404cbe262f0405d7e3192da63ee24a24b6a699b"}
82+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "6f8802cbd5053f88195deae0", "attempt_id": "ee7348e705c7", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:49.622450Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 600, "message_tokens": 512, "message_utf8_bytes": 1493, "cache_tensor_bytes": null, "serialized_payload_bytes": 1493, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10619.876333978027, "decode_payload_ms": 0.0, "receiver_prefill_ms": 107.23908396903425, "ttft_ms": 19.994916045106947, "decode_ms": 916.7372090741992}, "total_wall_ms": 1045.7014170242473, "ttft_ms": 19.994916045106947, "input_usage": {"receiver_prompt_tokens": 600, "sender_input_tokens": 2053, "sender_output_tokens": 512, "sender_prefill_ms": 348.635541042313, "sender_ttft_ms": 21.78091707173735, "sender_decode_ms": 10246.84870801866}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701105152, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6582726744, "mlx_cache_bytes": 3477464560, "process_rss_bytes": 6701105152}, "peak_prefill_memory": null, "peak_decode_memory": 6582726744, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['region'] != 'Sud' and row['units'] >= 2 and not row['returned'] and row['", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "a5360f49c84c5d88602af0cacb8eb14a906d47404d247d72e28a9c373feb8bd9", "output_hash": "ad6a28cb17112d61c6e0861769c09b6aca4ac8b11382ab844d0d76d38aeeeddd"}
83+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "dff5b8ff7f0b562672df6bac", "attempt_id": "4af52d4fb1cc", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:52.967207Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 1, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 165, "message_tokens": 67, "message_utf8_bytes": 238, "cache_tensor_bytes": null, "serialized_payload_bytes": 238, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2254.3724169954658, "decode_payload_ms": 0.0, "receiver_prefill_ms": 43.76500006765127, "ttft_ms": 20.686833071522415, "decode_ms": 18.83375004399568}, "total_wall_ms": 84.36074992641807, "ttft_ms": 20.686833071522415, "input_usage": {"receiver_prompt_tokens": 165, "sender_input_tokens": 2081, "sender_output_tokens": 85, "sender_prefill_ms": 492.3774159979075, "sender_ttft_ms": 25.297958054579794, "sender_decode_ms": 1734.3169170198962}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6701236224, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6314550016, "mlx_cache_bytes": 3624036480, "process_rss_bytes": 6701236224}, "peak_prefill_memory": null, "peak_decode_memory": 6314550016, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 1, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:NameError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c42d3913024ae8df3e3c52d4d39f2ffe452760aa6268f9e8a4cd162fdc0c1244", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
84+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c11304744694f15ee8ad5b33", "attempt_id": "072e89355803", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:55.240687Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 1}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 214, "message_tokens": 130, "message_utf8_bytes": 103, "cache_tensor_bytes": null, "serialized_payload_bytes": 103, "side_information_bytes": 0, "fetched_bytes": 269, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2144.001582986675, "decode_payload_ms": 0.0, "receiver_prefill_ms": 49.391667009331286, "ttft_ms": 20.21337498445064, "decode_ms": 926.9428750267252}, "total_wall_ms": 997.4543750286102, "ttft_ms": 20.21337498445064, "input_usage": {"receiver_prompt_tokens": 214, "sender_input_tokens": 2081, "sender_output_tokens": 85, "sender_prefill_ms": 341.33658302016556, "sender_ttft_ms": 22.545540938153863, "sender_decode_ms": 1777.7792499400675}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701236224, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6353527748, "mlx_cache_bytes": 3800132564, "process_rss_bytes": 6701236224}, "peak_prefill_memory": null, "peak_decode_memory": 6353527748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n price = row['unit_price'] * (1 - row['discount_pct'] / 100)\n if row['returned", "evidence_artifact": {"lookups": 1, "fetched_bytes": 269, "retrieved_refs": 1}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1ca01c6841ec89d2df6a744602238891e2a504aadf6d34de0cd09f8e803fe6dc", "output_hash": "1866ddef1765909117151568b3585703a3a42efe40e7c57f45cb199860548258"}
85+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "56ea44a15f71c558c2dced6e", "attempt_id": "caaced348dec", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:56.283336Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 5.879160636758321}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 2020, "message_tokens": 1937, "message_utf8_bytes": 8125, "cache_tensor_bytes": null, "serialized_payload_bytes": 1382, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.09729096200317144, "decode_payload_ms": 0.018749968148767948, "receiver_prefill_ms": 294.89733395166695, "ttft_ms": 14.12741607055068, "decode_ms": 1023.5364160034806}, "total_wall_ms": 1336.8795829592273, "ttft_ms": 14.12741607055068, "input_usage": {"receiver_prompt_tokens": 2020}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701236224, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7052050480, "mlx_cache_bytes": 3800132564, "process_rss_bytes": 6701236224}, "peak_prefill_memory": null, "peak_decode_memory": 7052050480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
86+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "da82041b0960ddc3a14c4229", "attempt_id": "6a38b839a652", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:58.001090Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6320513032}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 72843264, "serialized_payload_bytes": 72859340, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 333.8079579407349, "doc_prefill_ms": 309.61087497416884, "compress_ms": 2.533124992623925, "serialize_ms": 17.797374981455505, "load_ms": 2.209541038610041, "receiver_prefill_ms": 0.0, "query_prefill_ms": 49.67045795638114, "ttft_ms": 49.67045795638114, "decode_ms": 1017.886417103}, "total_wall_ms": 1071.5985000133514, "ttft_ms": 49.67045795638114, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6847250432, "backend_memory": {"mlx_active_bytes": 6397190152, "mlx_peak_bytes": 6449848672, "mlx_cache_bytes": 4718079109, "process_rss_bytes": 6847250432}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6449848672, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "698a195b26016f12bd71d6631fda0686907c1fdcb84c16df4296b26693b09e5c", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
87+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bbd615e60e849d8b1a7ba1ac", "attempt_id": "46733e8c5df8", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:59.142860Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6320513032}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "08df6c56032443d5", "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 72843264, "serialized_payload_bytes": 72859368, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 24.76350008510053, "doc_prefill_ms": 309.61087497416884, "compress_ms": 2.834332990460098, "serialize_ms": 21.69116702862084, "load_ms": 12.23729201592505, "receiver_prefill_ms": 0.0, "query_prefill_ms": 40.432416019029915, "ttft_ms": 40.432416019029915, "decode_ms": 1000.5399170331657}, "total_wall_ms": 1054.9829159863293, "ttft_ms": 40.432416019029915, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6920404992, "backend_memory": {"mlx_active_bytes": 6397190152, "mlx_peak_bytes": 6449848560, "mlx_cache_bytes": 4718079109, "process_rss_bytes": 6920404992}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6449848560, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2c13aad81bcd8dfdb14c874ddc3a115036f4e8d9db3242e3aa526e85a72a8286", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
88+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "d0d767d6b99935891bed0b25", "attempt_id": "cf27b3f98bac", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:00.269067Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6286408072}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 38697984, "serialized_payload_bytes": 38732174, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 26.633040979504585, "doc_prefill_ms": 309.61087497416884, "compress_ms": 13.843290973454714, "serialize_ms": 12.345083057880402, "load_ms": 2.3097090888768435, "receiver_prefill_ms": 0.0, "query_prefill_ms": 24.407291086390615, "ttft_ms": 24.407291086390615, "decode_ms": 1126.3775830157101}, "total_wall_ms": 1154.8897090833634, "ttft_ms": 24.407291086390615, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6958333952, "backend_memory": {"mlx_active_bytes": 6330349960, "mlx_peak_bytes": 6395258492, "mlx_cache_bytes": 5054269957, "process_rss_bytes": 6958333952}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6395258492, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "86fd69e1ab032c32583a0b7f11de3578bace1807d2de39b464d21bd2715100fa", "output_hash": "c03c392e692f4c667997049602f7e27418e41f00feeb390a815250c0569c1b13"}
89+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b0e0b0cdf8a9e8bf185f2ee9", "attempt_id": "e3d8c7bde859", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:01.493320Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268525576}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 20487168, "serialized_payload_bytes": 20521220, "side_information_bytes": 34052, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22.63975003734231, "doc_prefill_ms": 309.61087497416884, "compress_ms": 13.28904100228101, "serialize_ms": 8.920374908484519, "load_ms": 2.213416970334947, "receiver_prefill_ms": 0.0, "query_prefill_ms": 33.245792030356824, "ttft_ms": 33.245792030356824, "decode_ms": 1108.6429170100018}, "total_wall_ms": 1146.063124993816, "ttft_ms": 33.245792030356824, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6960431104, "backend_memory": {"mlx_active_bytes": 6294297608, "mlx_peak_bytes": 6354684588, "mlx_cache_bytes": 5121994885, "process_rss_bytes": 6960431104}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6354684588, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "user\nunknown\n the the the\n document\n document the\n the\n\n the The existing\n the\n\n\n the The\n\n\n the the the\n the the the, the the the the The the the The\n The the", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:IndentationError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ca347fd0215673b4a988d4223cdd6c3b690c3e1c9eb09e181f013f0778625d27", "output_hash": "c02cea32658f86423a17c812c900737ca47ebf057e88e9371d3fa6d2eff848a6"}
90+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "90260663756f15d8ab415e67", "attempt_id": "682b8fe93539", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:02.701867Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1976, "keep": 494, "compress_ms": 7.7032080153003335}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226732, "side_information_bytes": 15916, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 14.727665926329792, "doc_prefill_ms": 309.61087497416884, "compress_ms": 7.759916945360601, "serialize_ms": 6.733249989338219, "load_ms": 0.9612080175429583, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.56879199761897, "ttft_ms": 32.56879199761897, "decode_ms": 855.7253750041127}, "total_wall_ms": 891.3752499502152, "ttft_ms": 32.56879199761897, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 44}, "provider_cached_usage": null, "process_memory": 6964920320, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5153287301, "process_rss_bytes": 6964920320}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['status'] == 'completed':\n total_revenue += row['amount']\n return total_revenue", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "77876ae624b46c34b40953b0b294518e8694fcb2fbdff1665ed3871727ddf56f", "output_hash": "c1a9a621f1cfa430e5d57319c441f0b3699df1bdd0916d47d16bd8d3b636329d"}
91+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "faf3c5252bcbb25b131280a7", "attempt_id": "58c23501aa9f", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:03.653051Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1976, "keep": 494, "compress_ms": 6.551417056471109}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226726, "side_information_bytes": 15910, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 12.743625091388822, "doc_prefill_ms": 309.61087497416884, "compress_ms": 6.587499985471368, "serialize_ms": 5.920208059251308, "load_ms": 0.679041026160121, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.208042076788843, "ttft_ms": 29.208042076788843, "decode_ms": 932.1196250384673}, "total_wall_ms": 963.8700410723686, "ttft_ms": 29.208042076788843, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6979092480, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5153287301, "process_rss_bytes": 6979092480}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0\n for row in rows:\n if row['region'] != 'S' and row['region'] != 'Centre':\n total += row['units'] * row['price", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "4dbdad308845b08c8ddd8b1fd3d33a76db9d3416f55bc59b4a5b91ea60ddf302", "output_hash": "1b45a1a25e468a6175f2069e3767b3b695f311b3606ae8a8aa1c17d84cd9c6a7"}
92+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "5949397e740909d42f727519", "attempt_id": "06844a495723", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:04.665719Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1976, "keep": 494, "window_tokens": 44, "compress_ms": 58.248165994882584}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226772, "side_information_bytes": 15956, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 65.47891697846353, "doc_prefill_ms": 309.61087497416884, "compress_ms": 60.018207994289696, "serialize_ms": 5.194208002649248, "load_ms": 0.6166669772937894, "receiver_prefill_ms": 0.0, "query_prefill_ms": 23.563917027786374, "ttft_ms": 23.563917027786374, "decode_ms": 943.9251670846716}, "total_wall_ms": 1035.5242089135572, "ttft_ms": 23.563917027786374, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6979223552, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5256199293, "process_rss_bytes": 6979223552}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Shud' or len(r['units']) < 2:\n continue\n if r['", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c4eae48b1e15863a34266e4931e222e4c99ef00239336c8a4747c8ecf10da1ad", "output_hash": "eee59974af47160590a101092589ed6092783b0739bc4edfcc774b7d9be5d4cc"}
93+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4fc13be8620c3c81f55b6e3c", "attempt_id": "838ba2ff669e", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:05.814399Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1976, "keep": 494, "window_tokens": 19, "compress_ms": 58.98000008892268}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226771, "side_information_bytes": 15955, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 65.83225005306304, "doc_prefill_ms": 309.61087497416884, "compress_ms": 59.280334040522575, "serialize_ms": 6.3120829872787, "load_ms": 0.8338750340044498, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.883750015869737, "ttft_ms": 29.883750015869737, "decode_ms": 947.136375005357}, "total_wall_ms": 979.7340830555186, "ttft_ms": 29.883750015869737, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6979289088, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5266635901, "process_rss_bytes": 6979289088}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n rows = [r for r in rows if r['units'] >= 2 and r['region'] != 'S']\n rows = [r for r in rows if r['shipped_days']", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c2c7ecc64c9caf7ce1f6ad2de4912a25a929a2366bc0119fd9f498ffff62858a", "output_hash": "a18179203926d78856727d2c151d583a3bfd58bcc61dc42e63a72dbf44494cba"}
94+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c46cf3e234e112a0f5f8112b", "attempt_id": "41b243f2a98e", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:06.840622Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 72.07979203667492, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (44,974) and (1,16,44,949) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
95+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2ec276c68d8afd8d728c36d2", "attempt_id": "99963ae12b8e", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:06.959128Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1976, "keep": 988, "compress_ms": 8.043541922233999, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 19348992, "serialized_payload_bytes": 19382643, "side_information_bytes": 33651, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 25.42837499640882, "doc_prefill_ms": 309.61087497416884, "compress_ms": 16.244541970081627, "serialize_ms": 8.818165981210768, "load_ms": 1.8096249550580978, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.5376669857651, "ttft_ms": 30.5376669857651, "decode_ms": 1107.7149999327958}, "total_wall_ms": 1142.3345000948757, "ttft_ms": 30.5376669857651, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6999080960, "backend_memory": {"mlx_active_bytes": 6290392416, "mlx_peak_bytes": 6345574196, "mlx_cache_bytes": 5288831909, "process_rss_bytes": 6999080960}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6345574196, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "6cd4d317f609771874f10328ee8c9d0593582762cc86c71221870cbe34b03bf2", "output_hash": "11ef43e7a25672ac1b5bda0cb1f1c298234481cd58e721fb63925aba7d5c84b9"}
96+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ada2dd091bdea89a3739786c", "attempt_id": "19610da4e3c9", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:08.198628Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 60, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.01325004268437624, "decode_payload_ms": 0.0, "receiver_prefill_ms": 22.8534999769181, "ttft_ms": 19.033625023439527, "decode_ms": 18.955833977088332}, "total_wall_ms": 61.44804204814136, "ttft_ms": 19.033625023439527, "input_usage": {"receiver_prompt_tokens": 60}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6224191848, "mlx_cache_bytes": 131829180, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6224191848, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
97+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3cd7ce88f7932e2fe3a5dca6", "attempt_id": "ff9af49a194b", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:08.286310Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 3314, "message_tokens": 3244, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 15217, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011375057511031628, "decode_payload_ms": 0.0, "receiver_prefill_ms": 623.0147909373045, "ttft_ms": 22.94987498316914, "decode_ms": 757.846082909964}, "total_wall_ms": 1410.354790976271, "ttft_ms": 22.94987498316914, "input_usage": {"receiver_prompt_tokens": 3314}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054791664, "mlx_cache_bytes": 1964671360, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 7054791664, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
98+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e9ea83259bd11521d23b5992", "attempt_id": "cda8cd210413", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:09.724185Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 3244}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 591, "message_tokens": 516, "message_utf8_bytes": 2729, "cache_tensor_bytes": null, "serialized_payload_bytes": 2729, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 3.2423330703750253, "decode_payload_ms": 0.0, "receiver_prefill_ms": 101.38687491416931, "ttft_ms": 27.95499993953854, "decode_ms": 100.74745898600668}, "total_wall_ms": 231.90870904363692, "ttft_ms": 27.95499993953854, "input_usage": {"receiver_prompt_tokens": 591}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6564311128, "mlx_cache_bytes": 2357264768, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6564311128, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD pooling approach", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "11ac585095e23aad2dc25c5d250ea6fc439122cd288e6814f52c1cea040db97b", "output_hash": "b396b6ce678924171263f2c16423a6fb5ea85d772a5b7e43ad73b71e42b11ad7"}
99+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2c10d09f655cb2217d462ef3", "attempt_id": "fb2521d6b85c", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:09.990150Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 500, "n_sentences": 11}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 571, "message_tokens": 500, "message_utf8_bytes": 2058, "cache_tensor_bytes": null, "serialized_payload_bytes": 2058, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 4.259041976183653, "decode_payload_ms": 0.0, "receiver_prefill_ms": 102.35675005242229, "ttft_ms": 26.3324580155313, "decode_ms": 24.861750076524913}, "total_wall_ms": 155.5670410161838, "ttft_ms": 26.3324580155313, "input_usage": {"receiver_prompt_tokens": 571}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6552219736, "mlx_cache_bytes": 2704474496, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6552219736, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "a804cda2443cf12a08e11e226234585e0d1edf60d020061934488a33516d413b", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
100+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e5a31efff5f91b7eb69a8bc1", "attempt_id": "d3c27d3f5d40", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:10.174696Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 506, "n_sentences": 23}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 578, "message_tokens": 507, "message_utf8_bytes": 2495, "cache_tensor_bytes": null, "serialized_payload_bytes": 2495, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.6516250325366855, "decode_payload_ms": 0.0, "receiver_prefill_ms": 122.30041704606265, "ttft_ms": 17.13062508497387, "decode_ms": 417.9142080247402}, "total_wall_ms": 560.8353751013055, "ttft_ms": 17.13062508497387, "input_usage": {"receiver_prompt_tokens": 578}, "output_usage": {"output_tokens": 19}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6557036632, "mlx_cache_bytes": 3041952128, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6557036632, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension to VLAD where hard assignment is replaced with soft assignment for end-to-end training.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.27027027027027023, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "785023803d7f2c38c80debcdc8309ffa1c94e3c99227c52a36e7e66d830d2d9e", "output_hash": "e746ed4fb6926b79617f224cc269a459de798a399cfc39ed7c9b314e07455f73"}
101+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "7ee46ca5b5986914ad0f70a5", "attempt_id": "27e4e6a2dd08", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:16.132554Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 294, "message_tokens": 220, "message_utf8_bytes": 1044, "cache_tensor_bytes": null, "serialized_payload_bytes": 1044, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5371.418667025864, "decode_payload_ms": 0.0, "receiver_prefill_ms": 71.67716696858406, "ttft_ms": 19.78166599292308, "decode_ms": 70.39725000504404}, "total_wall_ms": 163.06179203093052, "ttft_ms": 19.78166599292308, "input_usage": {"receiver_prompt_tokens": 294, "sender_input_tokens": 3360, "sender_output_tokens": 220, "sender_prefill_ms": 656.0712079517543, "sender_ttft_ms": 15.746999997645617, "sender_decode_ms": 4695.432165986858}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6773899264, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6424421636, "mlx_cache_bytes": 3801168528, "process_rss_bytes": 6773899264}, "peak_prefill_memory": null, "peak_decode_memory": 6424421636, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3b08a9dd9882d778138853f9b9a6a55c7176ad67e2db289645e13b8fc1bac590", "output_hash": "4a81758367f5a2ca4e9aa1c032ddd94e54a15042d670c320c4955ecf300ae450"}
102+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "fa7aaf225eb9ea8ec9a07212", "attempt_id": "e68901764e49", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:24.829477Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 5, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 306, "message_tokens": 221, "message_utf8_bytes": 756, "cache_tensor_bytes": null, "serialized_payload_bytes": 756, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 8506.694042007439, "decode_payload_ms": 0.0, "receiver_prefill_ms": 55.022082990035415, "ttft_ms": 21.521249902434647, "decode_ms": 58.01579204853624}, "total_wall_ms": 135.69395896047354, "ttft_ms": 21.521249902434647, "input_usage": {"receiver_prompt_tokens": 306, "sender_input_tokens": 3388, "sender_output_tokens": 380, "sender_prefill_ms": 809.7484159516171, "sender_ttft_ms": 25.65774996764958, "sender_decode_ms": 7667.614832986146}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 6773932032, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6434170164, "mlx_cache_bytes": 4717179968, "process_rss_bytes": 6773932032}, "peak_prefill_memory": null, "peak_decode_memory": 6434170164, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "pooling method", "evidence_artifact": {"n_facts": 5, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bb6e01e5b80c84d0441a0b53d50795f440126a2d273b098a4adc01abc60175f", "output_hash": "8c7e3b1576d203644fb74ba50478132796f7da6038f8971317fa262590fd6536"}
103+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "fc7f1dec81af51ed135642f5", "attempt_id": "7ca101f1952a", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:33.574996Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 544, "message_tokens": 473, "message_utf8_bytes": 371, "cache_tensor_bytes": null, "serialized_payload_bytes": 371, "side_information_bytes": 0, "fetched_bytes": 1373, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 8585.026083979756, "decode_payload_ms": 0.0, "receiver_prefill_ms": 104.83375005424023, "ttft_ms": 22.171457996591926, "decode_ms": 62.47520900797099}, "total_wall_ms": 190.89829095173627, "ttft_ms": 22.171457996591926, "input_usage": {"receiver_prompt_tokens": 544, "sender_input_tokens": 3388, "sender_output_tokens": 380, "sender_prefill_ms": 719.3035829113796, "sender_ttft_ms": 25.42624995112419, "sender_decode_ms": 7836.96520794183}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 6773915648, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6535016536, "mlx_cache_bytes": 5052789824, "process_rss_bytes": 6773915648}, "peak_prefill_memory": null, "peak_decode_memory": 6535016536, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "pooling method", "evidence_artifact": {"lookups": 4, "fetched_bytes": 1373, "retrieved_refs": 4}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "15bd5c2c7fa5c1256d4044abb684ec5f8ec847c3e301d4402dc3bc463138b521", "output_hash": "8c7e3b1576d203644fb74ba50478132796f7da6038f8971317fa262590fd6536"}
104+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "9c0b36e54acecbc04d506a33", "attempt_id": "79213f5e6476", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:33.790608Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 2.8798258894776683}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 3314, "message_tokens": 3244, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 5284, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.09345798753201962, "decode_payload_ms": 0.02058397512882948, "receiver_prefill_ms": 704.8352500423789, "ttft_ms": 24.021124932914972, "decode_ms": 774.4359999196604}, "total_wall_ms": 1509.2177080223337, "ttft_ms": 24.021124932914972, "input_usage": {"receiver_prompt_tokens": 3314}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 6773915648, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054791664, "mlx_cache_bytes": 5052789824, "process_rss_bytes": 6773915648}, "peak_prefill_memory": null, "peak_decode_memory": 7054791664, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
105+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ee0a4bf8945963b7984eda04", "attempt_id": "7a5edbe6a2aa", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:35.959615Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6416064520}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 121024512, "serialized_payload_bytes": 121040621, "side_information_bytes": 16109, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 635.4355829535052, "doc_prefill_ms": 601.0837500216439, "compress_ms": 4.678208031691611, "serialize_ms": 23.705999949015677, "load_ms": 2.692666952498257, "receiver_prefill_ms": 0.0, "query_prefill_ms": 62.39358300808817, "ttft_ms": 62.39358300808817, "decode_ms": 771.4831250486895}, "total_wall_ms": 839.4441249547526, "ttft_ms": 62.39358300808817, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 7016480768, "backend_memory": {"mlx_active_bytes": 6539927560, "mlx_peak_bytes": 6590084908, "mlx_cache_bytes": 5976129213, "process_rss_bytes": 7016480768}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6590084908, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e72b7ab2934cfcf6888dcc2d829003003dc376d34928371486fae694c1a2098c", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
106+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "6d448ed450568948fe73b33b", "attempt_id": "962989a845d3", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:36.848715Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6416064520}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "35aaed2f1f78202e", "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 121024512, "serialized_payload_bytes": 121040649, "side_information_bytes": 16137, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 25.53833299316466, "doc_prefill_ms": 601.0837500216439, "compress_ms": 1.076459069736302, "serialize_ms": 24.271624977700412, "load_ms": 16.446624998934567, "receiver_prefill_ms": 0.0, "query_prefill_ms": 49.789458978921175, "ttft_ms": 49.789458978921175, "decode_ms": 747.1429159631953}, "total_wall_ms": 816.2833329988644, "ttft_ms": 49.789458978921175, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 7137984512, "backend_memory": {"mlx_active_bytes": 6539927560, "mlx_peak_bytes": 6590084796, "mlx_cache_bytes": 5976129213, "process_rss_bytes": 7137984512}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6590084796, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e188c41f1d9a05bc6e14927399c189fe21e3a4e72cabf24975d723a2d32d953f", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
107+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "1ac34c36ff3d7315fc796eae", "attempt_id": "d79add9587e3", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:37.723313Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6360621064}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 64294272, "serialized_payload_bytes": 64328508, "side_information_bytes": 34236, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 33.56612497009337, "doc_prefill_ms": 601.0837500216439, "compress_ms": 16.29454200156033, "serialize_ms": 16.862374963238835, "load_ms": 3.4387909108772874, "receiver_prefill_ms": 0.0, "query_prefill_ms": 132.63883395120502, "ttft_ms": 132.63883395120502, "decode_ms": 75.75041695963591}, "total_wall_ms": 214.81683303136379, "ttft_ms": 132.63883395120502, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7201554432, "backend_memory": {"mlx_active_bytes": 6426681352, "mlx_peak_bytes": 6493745728, "mlx_cache_bytes": 6267740861, "process_rss_bytes": 7201554432}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6493745728, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2a1a43b3fa4a399c34ba6ec7a1917e4c670fbe4963cb057ca7ec0551e351b23d", "output_hash": "75a872242c44bab0506befcac9b6a38c36a90ed87a8b8fbe27f7514cdf9dbb77"}
108+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ca17bd88ada65a286ee0a6f7", "attempt_id": "3187f01cea42", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:37.984179Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6329950216}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 34038144, "serialized_payload_bytes": 34072304, "side_information_bytes": 34160, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22.94916706159711, "doc_prefill_ms": 601.0837500216439, "compress_ms": 11.391667067073286, "serialize_ms": 11.177083011716604, "load_ms": 2.3093329509720206, "receiver_prefill_ms": 0.0, "query_prefill_ms": 143.6609170632437, "ttft_ms": 143.6609170632437, "decode_ms": 1126.990083954297}, "total_wall_ms": 1278.614499955438, "ttft_ms": 143.6609170632437, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7232307200, "backend_memory": {"mlx_active_bytes": 6366519304, "mlx_peak_bytes": 6424664597, "mlx_cache_bytes": 6436792893, "process_rss_bytes": 7232307200}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6424664597, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": " answer is is is the map is is the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the theuss the the the the", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1b95c993af3d21a934574f795df3e1fbf79b282247215a9eb48cbf2a848afa55", "output_hash": "2c0f9e0ae3f39ad1661d054a003adda4563e4914e846c6b20e1739fe08202a49"}
109+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0b133b77ac8a96e5c6b5bc99", "attempt_id": "72a86d76f83b", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:39.307416Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6325231624}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 3283, "keep": 821, "compress_ms": 8.413915988057852}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281293, "side_information_bytes": 15949, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19.421624951064587, "doc_prefill_ms": 601.0837500216439, "compress_ms": 8.4537910297513, "serialize_ms": 10.77312498819083, "load_ms": 1.2885419419035316, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.25900002103299, "ttft_ms": 30.25900002103299, "decode_ms": 44.62770896498114}, "total_wall_ms": 79.53341701067984, "ttft_ms": 30.25900002103299, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 7236452352, "backend_memory": {"mlx_active_bytes": 6357082120, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6446230077, "process_rss_bytes": 7236452352}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "203c88371d6551640470222a8f0db73f9cca2629b2d98351a486b0cc01f024f0", "output_hash": "aea88d526d067608708536c90844a71e2fdd73002205449a86f1441074ce02df"}
110+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "cecbdd65d043b664a2811af6", "attempt_id": "86745d804a03", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:39.430851Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6325231624}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 3283, "keep": 821, "compress_ms": 7.425334071740508}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281288, "side_information_bytes": 15944, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 16.988249961286783, "doc_prefill_ms": 601.0837500216439, "compress_ms": 7.470625103451312, "serialize_ms": 9.28033294621855, "load_ms": 0.7437919266521931, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.686542021110654, "ttft_ms": 32.686542021110654, "decode_ms": 219.86454201396555}, "total_wall_ms": 256.39504101127386, "ttft_ms": 32.686542021110654, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 11}, "provider_cached_usage": null, "process_memory": 7259455488, "backend_memory": {"mlx_active_bytes": 6357082120, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6446230077, "process_rss_bytes": 7259455488}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of VLAD with additional \"ghost\" clusters", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.588235294117647, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "70ea332320edce996b98525a13cea25477ebb6008fd834fc605091509e59c638", "output_hash": "2186a0d38b03b30a9f490d89c4029f93aed3afbdfe49ce3b728692172ec75684"}
111+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "34c653518a2822f0248089b7", "attempt_id": "f3b6dad3ff16", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:39.712620Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6294560776}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3283, "keep": 821, "window_tokens": 31, "compress_ms": 73.18974996451288}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281333, "side_information_bytes": 15989, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 83.91400007531047, "doc_prefill_ms": 601.0837500216439, "compress_ms": 76.17241598200053, "serialize_ms": 7.476332946680486, "load_ms": 0.6962079787626863, "receiver_prefill_ms": 0.0, "query_prefill_ms": 23.969750036485493, "ttft_ms": 23.969750036485493, "decode_ms": 469.9488329933956}, "total_wall_ms": 581.5955420257524, "ttft_ms": 23.969750036485493, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 23}, "provider_cached_usage": null, "process_memory": 7261257728, "backend_memory": {"mlx_active_bytes": 6358261768, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6516845253, "process_rss_bytes": 7261257728}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to improve aggregation of local features.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "47e80056bbc1a4eb0bd50907f4df60fa9febc185c53aa9bfb01a998dafb1fe62", "output_hash": "fecf4b3747739da00410fc6decc4ba4dde1d66f25ed88cb4a76dc7d3b1f7a46b"}
112+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "963b06874aa12c25afdf0c63", "attempt_id": "6b3a3d3bfe73", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:40.399242Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6325231624}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3283, "keep": 821, "window_tokens": 19, "compress_ms": 73.75320896971971}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281333, "side_information_bytes": 15989, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.70887504052371, "doc_prefill_ms": 601.0837500216439, "compress_ms": 73.89700005296618, "serialize_ms": 5.637041991576552, "load_ms": 1.3112499145790935, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.88529098406434, "ttft_ms": 31.88529098406434, "decode_ms": 456.10237505752593}, "total_wall_ms": 492.16875003185123, "ttft_ms": 31.88529098406434, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 22}, "provider_cached_usage": null, "process_memory": 7269089280, "backend_memory": {"mlx_active_bytes": 6358261768, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6539504325, "process_rss_bytes": 7269089280}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD approach is a pooling method used for language identification, inspired by recent work on speaker identification.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "47e80056bbc1a4eb0bd50907f4df60fa9febc185c53aa9bfb01a998dafb1fe62", "output_hash": "ba29f94ed96dd49c401ef0be023381ab5ee8b988ed76f46cdc19c5602ac9b29c"}
113+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f7d0a3fc2321b3c9704b89ec", "attempt_id": "d2a08aee8841", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:40.919206Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6294560776}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 87.07641693763435, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (31,1574) and (1,16,31,1533) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
114+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c08a83e61897f15f24d32a73", "attempt_id": "a9fbb7a566b2", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:41.064575Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6327222532}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 3283, "keep": 1642, "compress_ms": 8.948500035330653, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 32156928, "serialized_payload_bytes": 32191192, "side_information_bytes": 34264, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 33.74208300374448, "doc_prefill_ms": 601.0837500216439, "compress_ms": 20.669832942076027, "serialize_ms": 12.674166937358677, "load_ms": 2.0706249633803964, "receiver_prefill_ms": 0.0, "query_prefill_ms": 23.13250000588596, "ttft_ms": 23.13250000588596, "decode_ms": 386.1410419922322}, "total_wall_ms": 414.49383401777595, "ttft_ms": 23.13250000588596, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 17}, "provider_cached_usage": null, "process_memory": 7299727360, "backend_memory": {"mlx_active_bytes": 6361179844, "mlx_peak_bytes": 6420485060, "mlx_cache_bytes": 6590475617, "process_rss_bytes": 7299727360}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6420485060, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "A pooling technique for aggregating frame-level features into a single utterance-level feature.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "6f0cea2ea0fe134b9e5552bf881dec82625be5d2070aeab1427d56c909191a1b", "output_hash": "3ffdac3b266bbffe84697d9ce063315e72265a67e2c7d367e858dea2fc84ec04"}
added runs/raw/E0_smoke_llama.jsonl +114 −0
@@ -0,0 +1,114 @@
1+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "1acce7293fdf27d3784c6e81", "attempt_id": "3a91072adb5b", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:10.997570Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425501000}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 86, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 104.80358300264925, "ttft_ms": 38.55491697322577, "decode_ms": 42.36904205754399}, "total_wall_ms": 194.91720804944634, "ttft_ms": 38.55491697322577, "input_usage": {"receiver_prompt_tokens": 86}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6927237120, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6512662392, "mlx_cache_bytes": 88274304, "process_rss_bytes": 6927237120}, "peak_prefill_memory": null, "peak_decode_memory": 6512662392, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b764cdc0eab7137467211272fa539f1260d1bf2e71bcf6ff3bdc960f5c16aa14"}
2+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7e57ab223f53fe54106b58be", "attempt_id": "783d0cb42803", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.222377Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 91, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 49.940416938625276, "ttft_ms": 32.91054104920477, "decode_ms": 55.94099999871105}, "total_wall_ms": 139.85866599250585, "ttft_ms": 32.91054104920477, "input_usage": {"receiver_prompt_tokens": 91}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6927417344, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6516209100, "mlx_cache_bytes": 151221992, "process_rss_bytes": 6927417344}, "peak_prefill_memory": null, "peak_decode_memory": 6516209100, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
3+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "80eba4737155699c0a7c69f8", "attempt_id": "ef9db7ee8bcb", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.386669Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 91, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 60.74287497904152, "ttft_ms": 34.07541697379202, "decode_ms": 90.18166700843722}, "total_wall_ms": 185.84875005763024, "ttft_ms": 34.07541697379202, "input_usage": {"receiver_prompt_tokens": 91}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6927433728, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6516209100, "mlx_cache_bytes": 151221992, "process_rss_bytes": 6927433728}, "peak_prefill_memory": null, "peak_decode_memory": 6516209100, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
4+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0f310404290cf683416256cc", "attempt_id": "55ce2bb3b2ca", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.596095Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 88, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 53.37033304385841, "ttft_ms": 35.44983407482505, "decode_ms": 62.86750000435859}, "total_wall_ms": 152.5538330897689, "ttft_ms": 35.44983407482505, "input_usage": {"receiver_prompt_tokens": 88}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6927450112, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6519993804, "mlx_cache_bytes": 177665768, "process_rss_bytes": 6927450112}, "peak_prefill_memory": null, "peak_decode_memory": 6519993804, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
5+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "a06eb15f6c19a57624f42a03", "attempt_id": "8204848563f0", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.774406Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1862, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 444.0474580042064, "ttft_ms": 114.788957987912, "decode_ms": 270.48599999397993}, "total_wall_ms": 832.6459999661893, "ttft_ms": 114.788957987912, "input_usage": {"receiver_prompt_tokens": 1862}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6927777792, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7120900424, "mlx_cache_bytes": 884249596, "process_rss_bytes": 6927777792}, "peak_prefill_memory": null, "peak_decode_memory": 7120900424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
6+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b1d7b1cce8d3a973f5a29a1a", "attempt_id": "6bbc100b1afa", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:12.629456Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 495.7376670790836, "ttft_ms": 44.04479102231562, "decode_ms": 340.5430830316618}, "total_wall_ms": 884.0085410047323, "ttft_ms": 44.04479102231562, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6695288832, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 1354510628, "process_rss_bytes": 6695288832}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
7+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0d0757aa5d272f5e5f06f1f4", "attempt_id": "b5a52cd8b5e1", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:13.535016Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 451.73691597301513, "ttft_ms": 37.32558398041874, "decode_ms": 178.22666699066758}, "total_wall_ms": 670.3590000979602, "ttft_ms": 37.32558398041874, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6641074176, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 1354510628, "process_rss_bytes": 6641074176}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
8+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d29228b6041b9db070f72d7c", "attempt_id": "b70bcb139367", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:14.229625Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1864, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 440.22891600616276, "ttft_ms": 39.44133303593844, "decode_ms": 205.141541082412}, "total_wall_ms": 687.8513750853017, "ttft_ms": 39.44133303593844, "input_usage": {"receiver_prompt_tokens": 1864}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6551306240, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7159222632, "mlx_cache_bytes": 1594913060, "process_rss_bytes": 6551306240}, "peak_prefill_memory": null, "peak_decode_memory": 7159222632, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
9+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "4c3a0a4bad55488d3a6874e2", "attempt_id": "3170a08d5ad8", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:14.941063Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 617, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 177.61712498031557, "ttft_ms": 45.229541952721775, "decode_ms": 52.08924994803965}, "total_wall_ms": 276.7296659294516, "ttft_ms": 45.229541952721775, "input_usage": {"receiver_prompt_tokens": 617}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6554402816, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6867467768, "mlx_cache_bytes": 2050112196, "process_rss_bytes": 6554402816}, "peak_prefill_memory": null, "peak_decode_memory": 6867467768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
10+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d422269a403ca89ddb2d4079", "attempt_id": "0521e5fc82fd", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:15.236832Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 622, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 125.95404102467, "ttft_ms": 37.89616597350687, "decode_ms": 391.38012495823205}, "total_wall_ms": 556.9134170655161, "ttft_ms": 37.89616597350687, "input_usage": {"receiver_prompt_tokens": 622}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6554009600, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6878428684, "mlx_cache_bytes": 2363245688, "process_rss_bytes": 6554009600}, "peak_prefill_memory": null, "peak_decode_memory": 6878428684, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
11+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d1de1204b1dde86818bd5184", "attempt_id": "38fb4366138f", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:15.818841Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 622, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 168.85512496810406, "ttft_ms": 48.81137504708022, "decode_ms": 38.724916987121105}, "total_wall_ms": 258.002792019397, "ttft_ms": 48.81137504708022, "input_usage": {"receiver_prompt_tokens": 622}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6554058752, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6882262540, "mlx_cache_bytes": 2363245688, "process_rss_bytes": 6554058752}, "peak_prefill_memory": null, "peak_decode_memory": 6882262540, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
12+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "a97ce48716cccfffb0df7d46", "attempt_id": "8aff5e407615", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:16.096513Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 619, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 154.10387504380196, "ttft_ms": 35.48179194331169, "decode_ms": 60.69491698872298}, "total_wall_ms": 251.7090420005843, "ttft_ms": 35.48179194331169, "input_usage": {"receiver_prompt_tokens": 619}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6554173440, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6900022796, "mlx_cache_bytes": 2602517624, "process_rss_bytes": 6554173440}, "peak_prefill_memory": null, "peak_decode_memory": 6900022796, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
13+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "65b2e0d942958d4955094a38", "attempt_id": "168a1109b45c", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:16.370902Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 606, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 174.26633299328387, "ttft_ms": 25.856332969851792, "decode_ms": 169.48837495874614}, "total_wall_ms": 371.1075000464916, "ttft_ms": 25.856332969851792, "input_usage": {"receiver_prompt_tokens": 606}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6556090368, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6861684216, "mlx_cache_bytes": 2940961912, "process_rss_bytes": 6556090368}, "peak_prefill_memory": null, "peak_decode_memory": 6861684216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Haruki Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "55a01495263476a6da84eabe2c22899f74b3909040b9d9d4fe8a8b6431ff9e18"}
14+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2a898ab433c5394686322580", "attempt_id": "2dbaac58758b", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:16.760564Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 611, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 140.41587500832975, "ttft_ms": 38.561749970540404, "decode_ms": 40.86958395782858}, "total_wall_ms": 221.20654198806733, "ttft_ms": 38.561749970540404, "input_usage": {"receiver_prompt_tokens": 611}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6556106752, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6864387576, "mlx_cache_bytes": 3282191480, "process_rss_bytes": 6556106752}, "peak_prefill_memory": null, "peak_decode_memory": 6864387576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
15+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ec7b2a9fe84bf0e283f64e2a", "attempt_id": "4c277500c5b6", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:17.001906Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 611, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 169.2981249652803, "ttft_ms": 23.593208054080606, "decode_ms": 54.50175004079938}, "total_wall_ms": 248.86029202025384, "ttft_ms": 23.593208054080606, "input_usage": {"receiver_prompt_tokens": 611}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6557040640, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6864387576, "mlx_cache_bytes": 3282191480, "process_rss_bytes": 6557040640}, "peak_prefill_memory": null, "peak_decode_memory": 6864387576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
16+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0c1493bd8cd834eeb7aa0785", "attempt_id": "9169bdcba916", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:17.272578Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 608, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 180.0412500742823, "ttft_ms": 43.045250000432134, "decode_ms": 190.88037498295307}, "total_wall_ms": 415.3167080366984, "ttft_ms": 43.045250000432134, "input_usage": {"receiver_prompt_tokens": 608}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6557106176, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6893698552, "mlx_cache_bytes": 3438888056, "process_rss_bytes": 6557106176}, "peak_prefill_memory": null, "peak_decode_memory": 6893698552, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
17+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e9f693f84c29d8efb40e2291", "attempt_id": "79119d577b43", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:17.707911Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 511, "n_sentences": 28}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 608, "message_tokens": 511, "message_utf8_bytes": 1555, "cache_tensor_bytes": null, "serialized_payload_bytes": 1555, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.141499960795045, "decode_payload_ms": 0.0, "receiver_prefill_ms": 137.3269580071792, "ttft_ms": 42.00733406469226, "decode_ms": 276.81891701649874}, "total_wall_ms": 458.59983400441706, "ttft_ms": 42.00733406469226, "input_usage": {"receiver_prompt_tokens": 608}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6557253632, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6901202424, "mlx_cache_bytes": 3438888056, "process_rss_bytes": 6557253632}, "peak_prefill_memory": null, "peak_decode_memory": 6901202424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ff405ac03e5a10e14d572266d362d361b4219fa184f9be3bb806057879fd9ac", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
18+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "72b77475732181169f7e78f8", "attempt_id": "1910237ecabe", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:18.186661Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 29}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 610, "message_tokens": 508, "message_utf8_bytes": 1715, "cache_tensor_bytes": null, "serialized_payload_bytes": 1715, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.1243750341236591, "decode_payload_ms": 0.0, "receiver_prefill_ms": 195.8667910657823, "ttft_ms": 23.60216702800244, "decode_ms": 335.8849590877071}, "total_wall_ms": 557.7511669835076, "ttft_ms": 23.60216702800244, "input_usage": {"receiver_prompt_tokens": 610}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6557384704, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6864387576, "mlx_cache_bytes": 3438888056, "process_rss_bytes": 6557384704}, "peak_prefill_memory": null, "peak_decode_memory": 6864387576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2a8e20793db30181404c83877c2fa0ce0e9f4ce155842796b2e9a4db57e4d166", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
19+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3aa7131806a41f29a7e7635a", "attempt_id": "e3374140fba2", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:18.766348Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 512, "n_sentences": 29}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 614, "message_tokens": 512, "message_utf8_bytes": 1561, "cache_tensor_bytes": null, "serialized_payload_bytes": 1561, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.0653340723365545, "decode_payload_ms": 0.0, "receiver_prefill_ms": 152.72070898208767, "ttft_ms": 38.70024997740984, "decode_ms": 168.77012501936406}, "total_wall_ms": 362.51762497704476, "ttft_ms": 38.70024997740984, "input_usage": {"receiver_prompt_tokens": 614}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6557630464, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6897040888, "mlx_cache_bytes": 3597091960, "process_rss_bytes": 6557630464}, "peak_prefill_memory": null, "peak_decode_memory": 6897040888, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "a53881c8306c0d00c837f52a61f488dcff8e4df7502a72cc68c383b3156842b8", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
20+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2623ac986d4ae09b6f38cb6b", "attempt_id": "c118ee4ca756", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:19.148562Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 511, "n_sentences": 22}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 606, "message_tokens": 507, "message_utf8_bytes": 1893, "cache_tensor_bytes": null, "serialized_payload_bytes": 1893, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.0557500645518303, "decode_payload_ms": 0.0, "receiver_prefill_ms": 140.03133296500891, "ttft_ms": 41.58537497278303, "decode_ms": 201.92399993538857}, "total_wall_ms": 386.04374998249114, "ttft_ms": 41.58537497278303, "input_usage": {"receiver_prompt_tokens": 606}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6558023680, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6861684216, "mlx_cache_bytes": 3597091960, "process_rss_bytes": 6558023680}, "peak_prefill_memory": null, "peak_decode_memory": 6861684216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "22d6a96c1f0de138d69e1c361d6afdd8664290d33341c244479d5699bb0376d5", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
21+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c297d644a762a42131883b7d", "attempt_id": "bad721ed6bbf", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:36.584206Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 612, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 224.51616707257926, "ttft_ms": 51.759208901785314, "decode_ms": 58.00508300308138}, "total_wall_ms": 335.8493749983609, "ttft_ms": 51.759208901785314, "input_usage": {"receiver_prompt_tokens": 612, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6561136640, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6903856632, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561136640}, "peak_prefill_memory": null, "peak_decode_memory": 6903856632, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
22+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "af5ca197d146c72042e02b45", "attempt_id": "d46502500ded", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:36.941363Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 617, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 149.41387507133186, "ttft_ms": 42.637665988877416, "decode_ms": 313.3010419551283}, "total_wall_ms": 506.7510830704123, "ttft_ms": 42.637665988877416, "input_usage": {"receiver_prompt_tokens": 617, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6561169408, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6867467768, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561169408}, "peak_prefill_memory": null, "peak_decode_memory": 6867467768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 916-F, Fermont", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.28571428571428575}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "90f0dea38d92a987240fd60c09e83237873c207c6079538eed9a620045be71c5"}
23+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "55c02754131bfc6f507a1cf8", "attempt_id": "fe9605bbc93f", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:37.466533Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 617, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 199.5497919851914, "ttft_ms": 48.74649993143976, "decode_ms": 90.31466708984226}, "total_wall_ms": 339.81412497814745, "ttft_ms": 48.74649993143976, "input_usage": {"receiver_prompt_tokens": 617, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6561185792, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6867467768, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561185792}, "peak_prefill_memory": null, "peak_decode_memory": 6867467768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "68.6281", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "dcf770b51da1816894e8b51a64ce9116a511525b5b251eab5e28faebddc3df9f"}
24+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7cd74952809d0bb6cd5c0200", "attempt_id": "b25eefb17888", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:37.825977Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 614, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 189.33529197238386, "ttft_ms": 44.18208298739046, "decode_ms": 195.17187506426126}, "total_wall_ms": 430.00350007787347, "ttft_ms": 44.18208298739046, "input_usage": {"receiver_prompt_tokens": 614, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6561218560, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6904610296, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561218560}, "peak_prefill_memory": null, "peak_decode_memory": 6904610296, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
25+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "411f61984792a78bf25b5893", "attempt_id": "54a3e7485b18", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:57.740971Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 443, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 130.85904193576425, "ttft_ms": 37.889541941694915, "decode_ms": 35.92687495984137}, "total_wall_ms": 206.39716705773026, "ttft_ms": 37.889541941694915, "input_usage": {"receiver_prompt_tokens": 443, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6562988032, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6738705496, "mlx_cache_bytes": 5622825032, "process_rss_bytes": 6562988032}, "peak_prefill_memory": null, "peak_decode_memory": 6738705496, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
26+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3cc9446b3ad102c3baf6af25", "attempt_id": "e9c37c9c8d05", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:57.969766Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 448, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 144.1592500777915, "ttft_ms": 20.462832995690405, "decode_ms": 41.42437502741814}, "total_wall_ms": 207.5935839675367, "ttft_ms": 20.462832995690405, "input_usage": {"receiver_prompt_tokens": 448, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6563053568, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6741605464, "mlx_cache_bytes": 5873107016, "process_rss_bytes": 6563053568}, "peak_prefill_memory": null, "peak_decode_memory": 6741605464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
27+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c27ce71f958b1a8eab837012", "attempt_id": "34724b1a5730", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:58.201511Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 448, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 170.96658307127655, "ttft_ms": 44.05787494033575, "decode_ms": 87.86874997895211}, "total_wall_ms": 304.1407499695197, "ttft_ms": 44.05787494033575, "input_usage": {"receiver_prompt_tokens": 448, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6563119104, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6741605464, "mlx_cache_bytes": 5873107016, "process_rss_bytes": 6563119104}, "peak_prefill_memory": null, "peak_decode_memory": 6741605464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "58.0820", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b38eb8a0b5e545425ac83b4aa75ce5fda3dec19e0fb0d34757f1276e1e89f55b"}
28+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "04038b21771c154b41a48952", "attempt_id": "c059118314e3", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:58.528210Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 445, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 147.983834031038, "ttft_ms": 45.002790982834995, "decode_ms": 24.279332952573895}, "total_wall_ms": 218.7393750064075, "ttft_ms": 45.002790982834995, "input_usage": {"receiver_prompt_tokens": 445, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6563135488, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6762527832, "mlx_cache_bytes": 6031409224, "process_rss_bytes": 6563135488}, "peak_prefill_memory": null, "peak_decode_memory": 6762527832, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
29+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "84a6deed072a53ee196c6aee", "attempt_id": "3ee33f336d99", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:19.121617Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 918, "message_tokens": 821, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1496, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 220.54516593925655, "ttft_ms": 24.7326249955222, "decode_ms": 286.289207986556}, "total_wall_ms": 533.5963340476155, "ttft_ms": 24.7326249955222, "input_usage": {"receiver_prompt_tokens": 918, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6564102144, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6962990168, "mlx_cache_bytes": 6599819336, "process_rss_bytes": 6564102144}, "peak_prefill_memory": null, "peak_decode_memory": 6962990168, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1496, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "20e51e9edfce0f1f7bbb42564f4a801681629d693f39bdfd2f0fa18174b9872c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
30+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "9ae727a39e3113fae8cdea21", "attempt_id": "5576328117db", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:19.675066Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 927, "message_tokens": 825, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1498, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 253.4893749980256, "ttft_ms": 33.21612498257309, "decode_ms": 33.41083403211087}, "total_wall_ms": 321.7156248865649, "ttft_ms": 33.21612498257309, "input_usage": {"receiver_prompt_tokens": 927, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6564380672, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6967446616, "mlx_cache_bytes": 7055458376, "process_rss_bytes": 6564380672}, "peak_prefill_memory": null, "peak_decode_memory": 6967446616, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1498, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "dd58c6723f7a488b688a9800bd53ab4c41b6262ba448a078f439a766616620c8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
31+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "55543c366c72ffdbbd39c8af", "attempt_id": "66686a257fbb", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:20.017815Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 911, "message_tokens": 809, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1498, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 246.01679190527648, "ttft_ms": 32.576250028796494, "decode_ms": 151.59720892552286}, "total_wall_ms": 432.2824589908123, "ttft_ms": 32.576250028796494, "input_usage": {"receiver_prompt_tokens": 911, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6564429824, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6964415564, "mlx_cache_bytes": 7480819784, "process_rss_bytes": 6564429824}, "peak_prefill_memory": null, "peak_decode_memory": 6964415564, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1498, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e1819d55abb157660a30ec7e8741d5ff9892523ece3b510e1752a6800ca62402", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
32+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "99d6c56f377436971f9e4433", "attempt_id": "d7f4cc777047", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:20.470661Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 924, "message_tokens": 825, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1498, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 265.9725840203464, "ttft_ms": 36.55616601463407, "decode_ms": 28.996124980039895}, "total_wall_ms": 333.160417038016, "ttft_ms": 36.55616601463407, "input_usage": {"receiver_prompt_tokens": 924, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6564495360, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7001082968, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6564495360}, "peak_prefill_memory": null, "peak_decode_memory": 7001082968, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1498, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "dd58c6723f7a488b688a9800bd53ab4c41b6262ba448a078f439a766616620c8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7b6538619254e7b397e82909", "attempt_id": "499765337f31", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:20.830950Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1862, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 498.52987495251, "ttft_ms": 36.994458991102874, "decode_ms": 260.9179160790518}, "total_wall_ms": 800.1195830293, "ttft_ms": 36.994458991102874, "input_usage": {"receiver_prompt_tokens": 1862}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6569115648, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7120900424, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569115648}, "peak_prefill_memory": null, "peak_decode_memory": 7120900424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
34+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "93b4d8e4431a75f0233ad746", "attempt_id": "545588961097", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:21.651547Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 509.5061670290306, "ttft_ms": 49.28991699125618, "decode_ms": 315.8052080543712}, "total_wall_ms": 877.6191669749096, "ttft_ms": 49.28991699125618, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6569164800, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569164800}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
35+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "88d68cf77d7e11beadc7266c", "attempt_id": "2645dbd8e95d", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:22.553539Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 450.3524579340592, "ttft_ms": 31.306082964874804, "decode_ms": 152.0601250231266}, "total_wall_ms": 636.4715420641005, "ttft_ms": 31.306082964874804, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6569181184, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569181184}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
36+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e2b2bcfa03ca62ad479ecb48", "attempt_id": "f2ba1f71926d", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:23.209456Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1864, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 491.95970804430544, "ttft_ms": 44.978291960433125, "decode_ms": 222.31799992732704}, "total_wall_ms": 762.6348329940811, "ttft_ms": 44.978291960433125, "input_usage": {"receiver_prompt_tokens": 1864}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6569197568, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7179080028, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569197568}, "peak_prefill_memory": null, "peak_decode_memory": 7179080028, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
37+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "cc589ec7a25b40c391448878", "attempt_id": "8d1266ebf492", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:24.515315Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 6.305875023826957, "receiver_prefill_ms": 0.0, "query_prefill_ms": 72.47437501791865, "ttft_ms": 72.47437501791865, "decode_ms": 261.442834045738}, "total_wall_ms": 341.9492499670014, "ttft_ms": 72.47437501791865, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6991708160, "backend_memory": {"mlx_active_bytes": 7085356040, "mlx_peak_bytes": 7148451120, "mlx_cache_bytes": 8003223625, "process_rss_bytes": 6991708160}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7148451120, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
38+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "be00ef5059d78d7afef9989a", "attempt_id": "54c814cd5911", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:24.879707Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 5.109624937176704, "receiver_prefill_ms": 0.0, "query_prefill_ms": 77.48341606929898, "ttft_ms": 77.48341606929898, "decode_ms": 384.95275005698204}, "total_wall_ms": 469.2965829744935, "ttft_ms": 77.48341606929898, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6991921152, "backend_memory": {"mlx_active_bytes": 7086273544, "mlx_peak_bytes": 7152973104, "mlx_cache_bytes": 8294957129, "process_rss_bytes": 6991921152}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152973104, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
39+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "5dbb517dc1e39027d47ce53d", "attempt_id": "67c316ecb86b", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:25.371676Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 4.740834003314376, "receiver_prefill_ms": 0.0, "query_prefill_ms": 61.675542034208775, "ttft_ms": 61.675542034208775, "decode_ms": 163.4873750153929}, "total_wall_ms": 231.41041700728238, "ttft_ms": 61.675542034208775, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6991953920, "backend_memory": {"mlx_active_bytes": 7085356040, "mlx_peak_bytes": 7152973104, "mlx_cache_bytes": 8295874633, "process_rss_bytes": 6991953920}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152973104, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
40+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e30c2d93f6504db52f420ff5", "attempt_id": "201b24eb535f", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:25.623900Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 8.166083018295467, "receiver_prefill_ms": 0.0, "query_prefill_ms": 58.05266601964831, "ttft_ms": 58.05266601964831, "decode_ms": 214.86158296465874}, "total_wall_ms": 282.6327499933541, "ttft_ms": 58.05266601964831, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6992019456, "backend_memory": {"mlx_active_bytes": 7085356040, "mlx_peak_bytes": 7152416048, "mlx_cache_bytes": 8315551817, "process_rss_bytes": 6992019456}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152416048, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
41+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3bdce55ca1c203e6cb369146", "attempt_id": "cfdd9fae6058", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:25.999220Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 38.35595794953406, "receiver_prefill_ms": 0.0, "query_prefill_ms": 61.90962495747954, "ttft_ms": 61.90962495747954, "decode_ms": 312.5291250180453}, "total_wall_ms": 414.4973329966888, "ttft_ms": 61.90962495747954, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7202373632, "backend_memory": {"mlx_active_bytes": 7085192200, "mlx_peak_bytes": 7148139824, "mlx_cache_bytes": 8315715657, "process_rss_bytes": 7202373632}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7148139824, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
42+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "da49bbe0ee1937a1d6cea01b", "attempt_id": "262b74cca3b0", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:26.434479Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 27.08279201760888, "receiver_prefill_ms": 0.0, "query_prefill_ms": 36.74579202197492, "ttft_ms": 36.74579202197492, "decode_ms": 428.5999160492793}, "total_wall_ms": 493.9215829363093, "ttft_ms": 36.74579202197492, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7202406400, "backend_memory": {"mlx_active_bytes": 7086109704, "mlx_peak_bytes": 7152809264, "mlx_cache_bytes": 8314798153, "process_rss_bytes": 7202406400}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152809264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
43+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "16b714f92a451743570b3b31", "attempt_id": "cf8fc0d51247", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:26.948504Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 26.593707967549562, "receiver_prefill_ms": 0.0, "query_prefill_ms": 71.3852500775829, "ttft_ms": 71.3852500775829, "decode_ms": 116.4509579539299}, "total_wall_ms": 215.88545804843307, "ttft_ms": 71.3852500775829, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7202455552, "backend_memory": {"mlx_active_bytes": 7085192200, "mlx_peak_bytes": 7152809264, "mlx_cache_bytes": 8315715657, "process_rss_bytes": 7202455552}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152809264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
44+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3ab2375116fc39604b6bb3dd", "attempt_id": "0e62b672bf28", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:27.184808Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 29.1128340177238, "receiver_prefill_ms": 0.0, "query_prefill_ms": 56.22645898256451, "ttft_ms": 56.22645898256451, "decode_ms": 231.56520805787295}, "total_wall_ms": 318.4051669668406, "ttft_ms": 56.22645898256451, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7202471936, "backend_memory": {"mlx_active_bytes": 7085192200, "mlx_peak_bytes": 7152219440, "mlx_cache_bytes": 8315715657, "process_rss_bytes": 7202471936}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152219440, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
45+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "99a3adca6917879f925d6786", "attempt_id": "a920ac2a6176", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:27.586430Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 13.677041977643967, "receiver_prefill_ms": 0.0, "query_prefill_ms": 217.12250006385148, "ttft_ms": 217.12250006385148, "decode_ms": 343.93254097085446}, "total_wall_ms": 576.2448329478502, "ttft_ms": 217.12250006385148, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7316357120, "backend_memory": {"mlx_active_bytes": 6888567816, "mlx_peak_bytes": 6964742096, "mlx_cache_bytes": 8847362761, "process_rss_bytes": 7316357120}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6964742096, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
46+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8145f501bf31191588311a59", "attempt_id": "d17cb71f1d20", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:28.184216Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 3.7757500540465117, "receiver_prefill_ms": 0.0, "query_prefill_ms": 60.53770799189806, "ttft_ms": 60.53770799189806, "decode_ms": 339.0390001004562}, "total_wall_ms": 405.1002920605242, "ttft_ms": 60.53770799189806, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7316570112, "backend_memory": {"mlx_active_bytes": 6889485320, "mlx_peak_bytes": 6971168592, "mlx_cache_bytes": 9017721641, "process_rss_bytes": 7316570112}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6971168592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
47+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "9e09d4a66b95560c536e4bb6", "attempt_id": "3398867c7cff", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:28.612313Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 7.927541038952768, "receiver_prefill_ms": 0.0, "query_prefill_ms": 89.23725003842264, "ttft_ms": 89.23725003842264, "decode_ms": 179.2399160331115}, "total_wall_ms": 278.06874993257225, "ttft_ms": 89.23725003842264, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7316586496, "backend_memory": {"mlx_active_bytes": 6888502280, "mlx_peak_bytes": 6971171536, "mlx_cache_bytes": 9018704681, "process_rss_bytes": 7316586496}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6971171536, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
48+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c936609e91653935523c0908", "attempt_id": "43b1cfa6952a", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:28.911150Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 4.78758302051574, "receiver_prefill_ms": 0.0, "query_prefill_ms": 39.28579203784466, "ttft_ms": 39.28579203784466, "decode_ms": 184.25929197110236}, "total_wall_ms": 230.09091592393816, "ttft_ms": 39.28579203784466, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7316586496, "backend_memory": {"mlx_active_bytes": 6888567816, "mlx_peak_bytes": 6971415216, "mlx_cache_bytes": 9045216425, "process_rss_bytes": 7316586496}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6971415216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
49+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7d6acf26fc79182f5dbb1ed8", "attempt_id": "15136fb321a2", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:29.244120Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 3.18250001873821, "receiver_prefill_ms": 0.0, "query_prefill_ms": 185.57591596618295, "ttft_ms": 185.57591596618295, "decode_ms": 295.966292032972}, "total_wall_ms": 486.3574579358101, "ttft_ms": 185.57591596618295, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7371276288, "backend_memory": {"mlx_active_bytes": 6782612488, "mlx_peak_bytes": 6847246256, "mlx_cache_bytes": 9327865001, "process_rss_bytes": 7371276288}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6847246256, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
50+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c564af55b1354e855d13ebb5", "attempt_id": "d0c64cba8a9d", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:29.752314Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 2.182958065532148, "receiver_prefill_ms": 0.0, "query_prefill_ms": 63.19291703402996, "ttft_ms": 63.19291703402996, "decode_ms": 353.3098748885095}, "total_wall_ms": 420.24579201824963, "ttft_ms": 63.19291703402996, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7371325440, "backend_memory": {"mlx_active_bytes": 6782645256, "mlx_peak_bytes": 6854042320, "mlx_cache_bytes": 9327832233, "process_rss_bytes": 7371325440}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6854042320, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
51+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "5f2dc6cc5e94630f30047acf", "attempt_id": "ef05ade2e72f", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:30.194955Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 1.280625001527369, "receiver_prefill_ms": 0.0, "query_prefill_ms": 42.38108394201845, "ttft_ms": 42.38108394201845, "decode_ms": 163.35366701241583}, "total_wall_ms": 208.8556250091642, "ttft_ms": 42.38108394201845, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7371358208, "backend_memory": {"mlx_active_bytes": 6782579720, "mlx_peak_bytes": 6854042320, "mlx_cache_bytes": 9327897769, "process_rss_bytes": 7371358208}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6854042320, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
52+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "72933f8a71427c88fc872a97", "attempt_id": "4f258f174b74", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:30.427208Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 2.032542019151151, "receiver_prefill_ms": 0.0, "query_prefill_ms": 54.19116606935859, "ttft_ms": 54.19116606935859, "decode_ms": 194.77229204494506}, "total_wall_ms": 252.6729580713436, "ttft_ms": 54.19116606935859, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7371374592, "backend_memory": {"mlx_active_bytes": 6782579720, "mlx_peak_bytes": 6853991280, "mlx_cache_bytes": 9328159913, "process_rss_bytes": 7371374592}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6853991280, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
53+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "4a990a3bbc163af0254f4da9", "attempt_id": "decdff1d7a8a", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:30.835010Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 0.7832079427316785, "receiver_prefill_ms": 0.0, "query_prefill_ms": 48.5515829641372, "ttft_ms": 48.5515829641372, "decode_ms": 492.50808299984783}, "total_wall_ms": 543.5889170039445, "ttft_ms": 48.5515829641372, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 12}, "provider_cached_usage": null, "process_memory": 7392919552, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 9428331689, "process_rss_bytes": 7392919552}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "There is no Torsel-5 in the text.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "9692259f44dd18dd1cdbacd4f40fc5c86163138fa40a692cba37d5829d9efdae"}
54+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "edb167c9d4a110a48c5571db", "attempt_id": "9bd754b97ad0", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:31.403191Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 4.029666073620319, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.984708088450134, "ttft_ms": 59.984708088450134, "decode_ms": 240.85499998182058}, "total_wall_ms": 306.52774998452514, "ttft_ms": 59.984708088450134, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7392854016, "backend_memory": {"mlx_active_bytes": 6772323336, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9444846761, "process_rss_bytes": 7392854016}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
55+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "fd54709862c7fe9cda0b03ce", "attempt_id": "ae351568c289", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:31.732096Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 1.343625015579164, "receiver_prefill_ms": 0.0, "query_prefill_ms": 50.66633294336498, "ttft_ms": 50.66633294336498, "decode_ms": 127.93991691432893}, "total_wall_ms": 181.72441702336073, "ttft_ms": 50.66633294336498, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7392903168, "backend_memory": {"mlx_active_bytes": 6772192264, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9444977833, "process_rss_bytes": 7392903168}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not available.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "469147b9096ca76595ba03d6e8e849959829de7be9f36bbb9e437b35a34af4c5"}
56+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "f0e55404bf4c08124bf59b7e", "attempt_id": "5e01a4805fd3", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:31.936141Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 1.520541962236166, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.29674994852394, "ttft_ms": 31.29674994852394, "decode_ms": 252.26949993520975}, "total_wall_ms": 286.5531659917906, "ttft_ms": 31.29674994852394, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7392477184, "backend_memory": {"mlx_active_bytes": 6772192264, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 9444977833, "process_rss_bytes": 7392477184}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "AC-59L-3161", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "ad64332dcd39755f6897a634ba23cec1a004826ea0ea9dabd0093a9c2ec190b0"}
57+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "52c48b9797ca5c06552b1531", "attempt_id": "6091f1f0713c", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:32.302798Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 1.3935830211266875, "receiver_prefill_ms": 0.0, "query_prefill_ms": 57.18616594094783, "ttft_ms": 57.18616594094783, "decode_ms": 1790.9953750204295}, "total_wall_ms": 1851.3322500512004, "ttft_ms": 57.18616594094783, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7445053440, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 7445053440}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n\n\n\n\n\n\n\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n}\n\n}\n\n\n\n//}\n\n//////////////////////", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "ccccfe0de485e198bddd0417f4df11b4b4717882fdaf107fd56e690b6fa5f10b"}
58+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "24076303fc2fee00bc3a854e", "attempt_id": "a133016fcd07", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:34.175581Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 3.364000003784895, "receiver_prefill_ms": 0.0, "query_prefill_ms": 44.80479098856449, "ttft_ms": 44.80479098856449, "decode_ms": 1871.8788340920582}, "total_wall_ms": 1922.2292910562828, "ttft_ms": 44.80479098856449, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7445102592, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 7445102592}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n}\n\n}\n\n}\n\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "8a98bf1cdbfff3febaa313db2ee830212976ff44521e87d9f91743b75c411c7c"}
59+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "f481d30b9909473a1d70d9a6", "attempt_id": "dc9da1b7970d", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:36.118241Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 2.0380420610308647, "receiver_prefill_ms": 0.0, "query_prefill_ms": 19.68616701196879, "ttft_ms": 19.68616701196879, "decode_ms": 1811.1777500016615}, "total_wall_ms": 1834.7261250019073, "ttft_ms": 19.68616701196879, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7445102592, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 7445102592}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n//////////////////////////////////////////////////////////////////////////////////////////", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "9c1c85d0af791dd782b2fc598b4c2453d6625d60ad58bacf00b9ab5c4e254186"}
60+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c0bd23fcdf788f4e76591000", "attempt_id": "6bc2cb9bbadf", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:37.975107Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 5.221916944719851, "receiver_prefill_ms": 0.0, "query_prefill_ms": 48.19733300246298, "ttft_ms": 48.19733300246298, "decode_ms": 1727.6302080135792}, "total_wall_ms": 1782.6562089612707, "ttft_ms": 48.19733300246298, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6908248064, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 6908248064}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n//////////////////////////////////////////////////////////////////////////////////////////", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "9c1c85d0af791dd782b2fc598b4c2453d6625d60ad58bacf00b9ab5c4e254186"}
61+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c9bb5f57c39cd9b828c7092a", "attempt_id": "e9dc8ce0e20e", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:39.778208Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 36, "compress_ms": 223.24320790357888}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 245.0225839857012, "doc_prefill_ms": 437.422874965705, "compress_ms": 224.6504999930039, "serialize_ms": 20.149499992839992, "load_ms": 2.4289999855682254, "receiver_prefill_ms": 0.0, "query_prefill_ms": 38.21954189334065, "ttft_ms": 38.21954189334065, "decode_ms": 227.5533340871334}, "total_wall_ms": 515.092583023943, "ttft_ms": 38.21954189334065, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6921027584, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 10017287337, "process_rss_bytes": 6921027584}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
62+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ed4dca7d54c8d06691e35017", "attempt_id": "7f0d9603a35d", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:40.314694Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 162.85679198335856}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 192.2094589099288, "doc_prefill_ms": 437.422874965705, "compress_ms": 164.35245797038078, "serialize_ms": 27.638208004646003, "load_ms": 2.581541077233851, "receiver_prefill_ms": 0.0, "query_prefill_ms": 46.956875012256205, "ttft_ms": 46.956875012256205, "decode_ms": 314.66949998866767}, "total_wall_ms": 558.5637079784647, "ttft_ms": 46.956875012256205, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6911475712, "backend_memory": {"mlx_active_bytes": 6772323336, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10036014249, "process_rss_bytes": 6911475712}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
63+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "55d7c75cb9ebbc123fa690f9", "attempt_id": "957cb1da168c", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:40.897428Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 147.52570795826614}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 162.3099580174312, "doc_prefill_ms": 437.422874965705, "compress_ms": 149.2385419551283, "serialize_ms": 12.80104205943644, "load_ms": 1.1120840208604932, "receiver_prefill_ms": 0.0, "query_prefill_ms": 41.776582947932184, "ttft_ms": 41.776582947932184, "decode_ms": 141.10079209785908}, "total_wall_ms": 348.03975000977516, "ttft_ms": 41.776582947932184, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6913359872, "backend_memory": {"mlx_active_bytes": 6772192264, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10036145321, "process_rss_bytes": 6913359872}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
64+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e477b9a1d55a7ccb17c67451", "attempt_id": "4ae83690bd30", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:41.267533Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 38, "compress_ms": 129.34766698163003}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 146.5985419927165, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.98404195625335, "serialize_ms": 15.367292100563645, "load_ms": 2.997333067469299, "receiver_prefill_ms": 0.0, "query_prefill_ms": 51.74429202452302, "ttft_ms": 51.74429202452302, "decode_ms": 221.70766699127853}, "total_wall_ms": 425.10620795655996, "ttft_ms": 51.74429202452302, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6915211264, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 10043419817, "process_rss_bytes": 6915211264}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
65+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b46d4fcd3e2cacc4f4171eda", "attempt_id": "4d532bb7e02d", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:41.858815Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 0.9596669115126133, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.27637509815395, "ttft_ms": 27.27637509815395, "decode_ms": 24.090874940156937}, "total_wall_ms": 54.03216590639204, "ttft_ms": 27.27637509815395, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6770488328, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 10065439989, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
66+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d97a8ce2c81cfb32ae8dc73a", "attempt_id": "745244fed267", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:41.934154Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 1.297999988310039, "receiver_prefill_ms": 0.0, "query_prefill_ms": 52.7221669908613, "ttft_ms": 52.7221669908613, "decode_ms": 41.93408403079957}, "total_wall_ms": 97.69941691774875, "ttft_ms": 52.7221669908613, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10063867125, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
67+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "1899008e558dfaa91056372a", "attempt_id": "5a1814d043d3", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:42.050982Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 1.8834170186892152, "receiver_prefill_ms": 0.0, "query_prefill_ms": 44.49391597881913, "ttft_ms": 44.49391597881913, "decode_ms": 37.344375043176115}, "total_wall_ms": 85.45716595835984, "ttft_ms": 44.49391597881913, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10063867125, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
68+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8bf08c1d45fa158cabe9c701", "attempt_id": "e50e364f8914", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:42.158671Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 2.5057089515030384, "receiver_prefill_ms": 0.0, "query_prefill_ms": 35.62441701069474, "ttft_ms": 35.62441701069474, "decode_ms": 42.37329191528261}, "total_wall_ms": 82.29766692966223, "ttft_ms": 35.62441701069474, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6770488328, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 10065439989, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
69+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "6644725e8322781a5ad9d3f9", "attempt_id": "1dbb2113f66c", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:42.263727Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 36, "compress_ms": 155.98812501411885}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310621, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 179.18008391279727, "doc_prefill_ms": 437.422874965705, "compress_ms": 157.55825000815094, "serialize_ms": 21.383208106271923, "load_ms": 3.9268749533221126, "receiver_prefill_ms": 0.0, "query_prefill_ms": 872.3779579158872, "ttft_ms": 872.3779579158872, "decode_ms": 265.98583289887756}, "total_wall_ms": 1323.2823330909014, "ttft_ms": 872.3779579158872, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6851657728, "backend_memory": {"mlx_active_bytes": 6771045384, "mlx_peak_bytes": 6839888636, "mlx_cache_bytes": 10584059665, "process_rss_bytes": 6851657728}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6839888636, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2b623b234e50bb45b3d64ddb4e5fc624d2e8d987140d846139a2f1fc5933cbce", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
70+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "bbc8aae14960288c98be5b42", "attempt_id": "1dc7fe1a4905", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:43.609212Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 154.456916032359}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310619, "side_information_bytes": 12891, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 177.84450005274266, "doc_prefill_ms": 437.422874965705, "compress_ms": 156.14125004503876, "serialize_ms": 21.469792001880705, "load_ms": 1.3804170303046703, "receiver_prefill_ms": 0.0, "query_prefill_ms": 938.1341250846162, "ttft_ms": 938.1341250846162, "decode_ms": 359.9463750142604}, "total_wall_ms": 1478.9378329878673, "ttft_ms": 938.1341250846162, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6911180800, "backend_memory": {"mlx_active_bytes": 6771897352, "mlx_peak_bytes": 6844136300, "mlx_cache_bytes": 10614410077, "process_rss_bytes": 6911180800}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6844136300, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "848149b18f72edc487ee684ea55641b98ac2d675e22c288ffa0f3a81f652f67e", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
71+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "5f17d37ee4da2fa8a0c22e08", "attempt_id": "1b3d099d25e4", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:45.112131Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 129.18295804411173}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310621, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 145.54508298169822, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.7377500925213, "serialize_ms": 14.53899999614805, "load_ms": 1.1376250768080354, "receiver_prefill_ms": 0.0, "query_prefill_ms": 1037.930167047307, "ttft_ms": 1037.930167047307, "decode_ms": 149.28004110697657}, "total_wall_ms": 1335.5639590881765, "ttft_ms": 1037.930167047307, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6947028992, "backend_memory": {"mlx_active_bytes": 6771274760, "mlx_peak_bytes": 6844136300, "mlx_cache_bytes": 10615032669, "process_rss_bytes": 6947028992}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6844136300, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2b623b234e50bb45b3d64ddb4e5fc624d2e8d987140d846139a2f1fc5933cbce", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
72+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "fc3835ebfc431e41ae846d63", "attempt_id": "9f4834358d02", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:46.471228Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 38, "compress_ms": 139.58499999716878}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310621, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 156.92758397199214, "doc_prefill_ms": 437.422874965705, "compress_ms": 141.24699996318668, "serialize_ms": 15.446165925823152, "load_ms": 1.5408750623464584, "receiver_prefill_ms": 0.0, "query_prefill_ms": 883.123290957883, "ttft_ms": 883.123290957883, "decode_ms": 175.33191700931638}, "total_wall_ms": 1218.847917043604, "ttft_ms": 883.123290957883, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6989185024, "backend_memory": {"mlx_active_bytes": 6771012616, "mlx_peak_bytes": 6841477948, "mlx_cache_bytes": 10621881181, "process_rss_bytes": 6989185024}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6841477948, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2b623b234e50bb45b3d64ddb4e5fc624d2e8d987140d846139a2f1fc5933cbce", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
73+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8effa93c0f06a54d68593ab3", "attempt_id": "b758fe81f510", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:47.848242Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 3.6407920997589827, "receiver_prefill_ms": 0.0, "query_prefill_ms": 36.66383307427168, "ttft_ms": 36.66383307427168, "decode_ms": 99.13308289833367}, "total_wall_ms": 141.15158305503428, "ttft_ms": 36.66383307427168, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7049773056, "backend_memory": {"mlx_active_bytes": 6776222728, "mlx_peak_bytes": 6840512432, "mlx_cache_bytes": 10683583325, "process_rss_bytes": 7049773056}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6840512432, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not found.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "0802559db1375af3ff5caabba71acea1d6299f1a7fc64b6a5024f19cbd33b72f"}
74+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d4eb8890a5e642f7ab6000aa", "attempt_id": "d61c0956fdb3", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:48.013724Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 3.137958934530616, "receiver_prefill_ms": 0.0, "query_prefill_ms": 52.22874996252358, "ttft_ms": 52.22874996252358, "decode_ms": 385.93683403450996}, "total_wall_ms": 442.7836670074612, "ttft_ms": 52.22874996252358, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7049789440, "backend_memory": {"mlx_active_bytes": 6778057736, "mlx_peak_bytes": 6836265264, "mlx_cache_bytes": 10681748317, "process_rss_bytes": 7049789440}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6836265264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
75+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2d065ff453763a5f4c3731f7", "attempt_id": "999a8ae40b7c", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:48.479776Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 6.236125016584992, "receiver_prefill_ms": 0.0, "query_prefill_ms": 53.27745899558067, "ttft_ms": 53.27745899558067, "decode_ms": 147.6881660055369}, "total_wall_ms": 208.8929999154061, "ttft_ms": 53.27745899558067, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7049805824, "backend_memory": {"mlx_active_bytes": 6776222728, "mlx_peak_bytes": 6836265264, "mlx_cache_bytes": 10683583325, "process_rss_bytes": 7049805824}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6836265264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
76+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0ed88b91f50021037e7b63fa", "attempt_id": "10349d5e5002", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:48.711103Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 2.9613750521093607, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.615874962881207, "ttft_ms": 31.615874962881207, "decode_ms": 254.29916696157306}, "total_wall_ms": 290.79958400689065, "ttft_ms": 31.615874962881207, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7049822208, "backend_memory": {"mlx_active_bytes": 6776222728, "mlx_peak_bytes": 6837017456, "mlx_cache_bytes": 10683583325, "process_rss_bytes": 7049822208}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6837017456, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
77+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d9b75cee222d17ea15050553", "attempt_id": "670fbabb21dc", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:49.621118Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 95, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.00925001222640276, "decode_payload_ms": 0.0, "receiver_prefill_ms": 63.3784169331193, "ttft_ms": 51.294708042405546, "decode_ms": 1745.2523339306936}, "total_wall_ms": 1860.6292499462143, "ttft_ms": 51.294708042405546, "input_usage": {"receiver_prompt_tokens": 95}, "output_usage": {"output_tokens": 47}, "provider_cached_usage": null, "process_memory": 6802898944, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6519191004, "mlx_cache_bytes": 388432592, "process_rss_bytes": 6802898944}, "peak_prefill_memory": null, "peak_decode_memory": 6519191004, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n if rows < 0:\n return \"Invalid number of rows\"\n elif rows == 0:\n return 0\n else:\n return rows * 100\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "TypeError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "06087c1270f1bbd4d0fcf8573af49de69f8e400a5022dd76dbda1cb50c96ba93"}
78+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d78132a69217e36ae518ef36", "attempt_id": "60b3015340ec", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:51.527146Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 1898, "message_tokens": 1793, "message_utf8_bytes": 7944, "cache_tensor_bytes": null, "serialized_payload_bytes": 7944, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.01525005791336298, "decode_payload_ms": 0.0, "receiver_prefill_ms": 470.90358298737556, "ttft_ms": 45.215333928354084, "decode_ms": 4239.911125041544}, "total_wall_ms": 4759.3687500339, "ttft_ms": 45.215333928354084, "input_usage": {"receiver_prompt_tokens": 1898}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6803439616, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7130223064, "mlx_cache_bytes": 869458036, "process_rss_bytes": 6803439616}, "peak_prefill_memory": null, "peak_decode_memory": 7130223064, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
79+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e08997cc2feb77c2c937b94a", "attempt_id": "edd940dbeb54", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:56.333998Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1793}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 625, "message_tokens": 515, "message_utf8_bytes": 2185, "cache_tensor_bytes": null, "serialized_payload_bytes": 2185, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.051000017672777, "decode_payload_ms": 0.0, "receiver_prefill_ms": 181.25262495595962, "ttft_ms": 51.436083973385394, "decode_ms": 3500.617416924797}, "total_wall_ms": 3734.8810830153525, "ttft_ms": 51.436083973385394, "input_usage": {"receiver_prompt_tokens": 625}, "output_usage": {"output_tokens": 95}, "provider_cached_usage": null, "process_memory": 6804488192, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6872104472, "mlx_cache_bytes": 1329424948, "process_rss_bytes": 6804488192}, "peak_prefill_memory": null, "peak_decode_memory": 6872104472, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n rows = [row for row in rows if row['region'] != 'Sud' and row['returned'] == False and row['units'] >= 2]\n revenue = sum(row['units'] * row['unit_price'] * (1 - row['discount_pct'] / 100) for row in rows)\n revenue *= 1.09975\n return round(revenue, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2413b1319a63f1b6cd0f54d110715d18bcdb9bb119470d5b741d8c0d7ebff49c", "output_hash": "d6ad48d19daa4028fb032076e444c54864be0c65981308d25e8463d4e1012ab9"}
80+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ddf8979de181cebec5f1827f", "attempt_id": "1c2fdfbd9e2b", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:00.118142Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 501, "n_sentences": 6}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 604, "message_tokens": 498, "message_utf8_bytes": 1553, "cache_tensor_bytes": null, "serialized_payload_bytes": 1553, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.8467919435352087, "decode_payload_ms": 0.0, "receiver_prefill_ms": 181.44291604403406, "ttft_ms": 17.23441598005593, "decode_ms": 3289.129166980274}, "total_wall_ms": 3489.383792039007, "ttft_ms": 17.23441598005593, "input_usage": {"receiver_prompt_tokens": 604}, "output_usage": {"output_tokens": 93}, "provider_cached_usage": null, "process_memory": 6804537344, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6860832280, "mlx_cache_bytes": 1689692724, "process_rss_bytes": 6804537344}, "peak_prefill_memory": null, "peak_decode_memory": 6860832280, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0\n for r in rows:\n if not r['returned'] and r['region'] != 'Sud' and r['units'] >= 2:\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total * (1 + 0.09975), 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "48e30493b93ebcda2339038b160af9a3c10308d66882e2c52b74d425d38bc3ec", "output_hash": "0c7889ee8e7cfd64c00351584d70e550d93bdc704007cedf1a26ae62d0807c38"}
81+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d7584634c71bf21e0d6eecd1", "attempt_id": "70b30cfac8e5", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:03.652108Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 512, "n_sentences": 25}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 618, "message_tokens": 512, "message_utf8_bytes": 2405, "cache_tensor_bytes": null, "serialized_payload_bytes": 2405, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.066208933480084, "decode_payload_ms": 0.0, "receiver_prefill_ms": 154.62425001896918, "ttft_ms": 39.64283294044435, "decode_ms": 3531.6903749480844}, "total_wall_ms": 3728.690249961801, "ttft_ms": 39.64283294044435, "input_usage": {"receiver_prompt_tokens": 618}, "output_usage": {"output_tokens": 98}, "provider_cached_usage": null, "process_memory": 6804602880, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6868844056, "mlx_cache_bytes": 2035198516, "process_rss_bytes": 6804602880}, "peak_prefill_memory": null, "peak_decode_memory": 6868844056, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for r in rows:\n if r['region'] != 'Sud' and r['units'] >= 2:\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100)\n total_revenue += revenue\n tax = total_revenue * 0.09975\n return round(total_revenue + tax, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1b918631e0fb86c8eeb18ec2fd014d346b49af210fb4d83a68e05ad9997e0880", "output_hash": "386afcb0ebee66414bfab34fc6c287fa4764e3c3d58a941f153b33b00abcdd81"}
82+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "1da1f33c7f89ce4370a99fc6", "attempt_id": "be4c357385b8", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:27.648986Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 622, "message_tokens": 512, "message_utf8_bytes": 1692, "cache_tensor_bytes": null, "serialized_payload_bytes": 1692, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20225.024625076912, "decode_payload_ms": 0.0, "receiver_prefill_ms": 150.09825001470745, "ttft_ms": 49.69558399170637, "decode_ms": 3554.733332945034}, "total_wall_ms": 3756.172207999043, "ttft_ms": 49.69558399170637, "input_usage": {"receiver_prompt_tokens": 622, "sender_input_tokens": 1927, "sender_output_tokens": 512, "sender_prefill_ms": 470.2747920528054, "sender_ttft_ms": 45.99579190835357, "sender_decode_ms": 19706.098709022626}, "output_usage": {"output_tokens": 96}, "provider_cached_usage": null, "process_memory": 6805078016, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6902070808, "mlx_cache_bytes": 3238791244, "process_rss_bytes": 6805078016}, "peak_prefill_memory": null, "peak_decode_memory": 6902070808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total_revenue = sum(\n (row['units'] * row['unit_price'] * (1 - row['discount_pct'] / 100)) \n for row in rows if not row['returned'] and row['region'] != 'Sud'\n )\n total_revenue *= 1.09975 # Apply 9.975% tax\n return round(total_revenue, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.8, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "ce216383da3fca3914a369e3532324ff4046ab2dfdd40ea6c9eb2f590604f161", "output_hash": "2f4e3ebeefa8303d0e566703fe6dc53c934833e5434ecb87e0da10191df67a70"}
83+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "6ac733142fc67d77e042f09a", "attempt_id": "c339466219ca", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:51.782152Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 5, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 526, "message_tokens": 405, "message_utf8_bytes": 1333, "cache_tensor_bytes": null, "serialized_payload_bytes": 1333, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20329.91862494964, "decode_payload_ms": 0.0, "receiver_prefill_ms": 156.54124994762242, "ttft_ms": 29.017125023528934, "decode_ms": 2839.151250082068}, "total_wall_ms": 3026.1665829457343, "ttft_ms": 29.017125023528934, "input_usage": {"receiver_prompt_tokens": 526, "sender_input_tokens": 1957, "sender_output_tokens": 512, "sender_prefill_ms": 517.0142910210416, "sender_ttft_ms": 44.45845796726644, "sender_decode_ms": 19765.974750043824}, "output_usage": {"output_tokens": 78}, "provider_cached_usage": null, "process_memory": 6805225472, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6815022616, "mlx_cache_bytes": 4044957404, "process_rss_bytes": 6805225472}, "peak_prefill_memory": null, "peak_decode_memory": 6815022616, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)", "evidence_artifact": {"n_facts": 5, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f05cb6c2f9bf2f73fb1dcd0e0639f005f621b57c0475151370086fbe11e43e46", "output_hash": "20782f8c8645c8bef359993e2e9d5715b1b1a2f52039b25419491a0f11425629"}
84+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "87cd1fcac8bca76f19512355", "attempt_id": "f11e12cb7f24", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:14.870978Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 5}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 646, "message_tokens": 540, "message_utf8_bytes": 432, "cache_tensor_bytes": null, "serialized_payload_bytes": 432, "side_information_bytes": 0, "fetched_bytes": 1265, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20015.72812499944, "decode_payload_ms": 0.0, "receiver_prefill_ms": 220.63470794819295, "ttft_ms": 38.1325000198558, "decode_ms": 2624.743292108178}, "total_wall_ms": 2885.4318329831585, "ttft_ms": 38.1325000198558, "input_usage": {"receiver_prompt_tokens": 646, "sender_input_tokens": 1957, "sender_output_tokens": 512, "sender_prefill_ms": 533.8998339138925, "sender_ttft_ms": 45.12329096905887, "sender_decode_ms": 19434.01129206177}, "output_usage": {"output_tokens": 73}, "provider_cached_usage": null, "process_memory": 6805356544, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6884572780, "mlx_cache_bytes": 4429656304, "process_rss_bytes": 6805356544}, "peak_prefill_memory": null, "peak_decode_memory": 6884572780, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned'] or r['region'] == 'Sud' or r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'])\n return total", "evidence_artifact": {"lookups": 5, "fetched_bytes": 1265, "retrieved_refs": 5}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "60edc4dbc168ed8b9a202ed54bacfb9f509e83f39b892b5d729d21f65d514068", "output_hash": "11e6c0f73669642380a1848864610e25be94ddbc954c1891d8be076ad8de2943"}
85+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "612cacc00b9ad234007dcb5f", "attempt_id": "477199f5cbef", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:17.807723Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 6.139103554868624}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 1898, "message_tokens": 1793, "message_utf8_bytes": 7944, "cache_tensor_bytes": null, "serialized_payload_bytes": 1294, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.10441604536026716, "decode_payload_ms": 0.018208986148238182, "receiver_prefill_ms": 511.64245803374797, "ttft_ms": 22.179667023010552, "decode_ms": 4115.618415991776}, "total_wall_ms": 4653.617207892239, "ttft_ms": 22.179667023010552, "input_usage": {"receiver_prompt_tokens": 1898}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6805389312, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7130223064, "mlx_cache_bytes": 4429656304, "process_rss_bytes": 6805389312}, "peak_prefill_memory": null, "peak_decode_memory": 7130223064, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
86+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0d74956e2bab90a3256d5dd2", "attempt_id": "463267041c39", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:23.058543Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6873412616}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 212516864, "serialized_payload_bytes": 212529870, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 547.9008340043947, "doc_prefill_ms": 491.1570419790223, "compress_ms": 5.470708012580872, "serialize_ms": 47.444583964534104, "load_ms": 4.824541974812746, "receiver_prefill_ms": 0.0, "query_prefill_ms": 73.48570798058063, "ttft_ms": 73.48570798058063, "decode_ms": 4921.614417107776}, "total_wall_ms": 5001.988791045733, "ttft_ms": 73.48570798058063, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6697304064, "backend_memory": {"mlx_active_bytes": 7103706120, "mlx_peak_bytes": 7162344676, "mlx_cache_bytes": 7737651801, "process_rss_bytes": 6697304064}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 7162344676, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "522bffe332074bcb796f108c4005f47eec780836ba66d0efb9b603d20188ccef", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
87+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "74d9df9a04f363079fbb80ec", "attempt_id": "e3be8f7bcc14", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:28.490509Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6873248776}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "acd4936815db7cad", "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 212516864, "serialized_payload_bytes": 212529898, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 373.191750026308, "doc_prefill_ms": 491.1570419790223, "compress_ms": 224.6105830417946, "serialize_ms": 148.3124999795109, "load_ms": 52.63670801650733, "receiver_prefill_ms": 0.0, "query_prefill_ms": 69.78820904623717, "ttft_ms": 69.78820904623717, "decode_ms": 4938.05329198949}, "total_wall_ms": 5062.4504160368815, "ttft_ms": 69.78820904623717, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 7312818176, "backend_memory": {"mlx_active_bytes": 7103542280, "mlx_peak_bytes": 7162180836, "mlx_cache_bytes": 7737815641, "process_rss_bytes": 7312818176}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 7162180836, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "6c67a21bc7e0c6e1c50b732378ea78ab585671684a96ad5d43a4f97a0218becf", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
88+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0ebc2d3b95b910b895dba16b", "attempt_id": "edb994f3389f", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:33.691256Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6774158344}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 112899584, "serialized_payload_bytes": 112926716, "side_information_bytes": 27132, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 89.32725002523512, "doc_prefill_ms": 491.1570419790223, "compress_ms": 38.98474993184209, "serialize_ms": 50.01079197973013, "load_ms": 6.6009999718517065, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.833958046510816, "ttft_ms": 59.833958046510816, "decode_ms": 4563.0839159712195}, "total_wall_ms": 4631.997165968642, "ttft_ms": 59.833958046510816, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6839533568, "backend_memory": {"mlx_active_bytes": 6897103880, "mlx_peak_bytes": 6978903204, "mlx_cache_bytes": 9336604729, "process_rss_bytes": 6839533568}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6978903204, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e7a89df0990ce3437929cd0e918adccbe4bf4185ee218535531bb40e7ff755a6", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
89+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2e3064455744be4e77fe8483", "attempt_id": "c03ae7602083", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:38.493748Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 59770368, "serialized_payload_bytes": 59797425, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 122.98837502021343, "doc_prefill_ms": 491.1570419790223, "compress_ms": 90.87250009179115, "serialize_ms": 31.734207994304597, "load_ms": 4.436874995008111, "receiver_prefill_ms": 0.0, "query_prefill_ms": 45.52912502549589, "ttft_ms": 45.52912502549589, "decode_ms": 4216.681290999986}, "total_wall_ms": 4269.118999945931, "ttft_ms": 45.52912502549589, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 108}, "provider_cached_usage": null, "process_memory": 6784548864, "backend_memory": {"mlx_active_bytes": 6786085896, "mlx_peak_bytes": 6859957796, "mlx_cache_bytes": 9791039545, "process_rss_bytes": 6784548864}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6859957796, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n tax = total * 0.09975\n total += tax\n return round(total, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "a2b102a2f28b534d74058a7540f8b4289c47979a55e3bc77a025b8b925612e15", "output_hash": "b9fa353e4e3364bf240cf377ea132be87c593ad7da514a543129c39d1b5e887a"}
90+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "efb3addedafe4789379451f0", "attempt_id": "a52d9e1663a8", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:42.906098Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1853, "keep": 463, "compress_ms": 54.579207906499505}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113427, "side_information_bytes": 12883, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 88.03220803383738, "doc_prefill_ms": 491.1570419790223, "compress_ms": 54.63979090563953, "serialize_ms": 33.18020899314433, "load_ms": 2.2533750161528587, "receiver_prefill_ms": 0.0, "query_prefill_ms": 37.718416075222194, "ttft_ms": 37.718416075222194, "decode_ms": 3739.06579206232}, "total_wall_ms": 3782.042583101429, "ttft_ms": 37.718416075222194, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 100}, "provider_cached_usage": null, "process_memory": 6756958208, "backend_memory": {"mlx_active_bytes": 6783333384, "mlx_peak_bytes": 6819935468, "mlx_cache_bytes": 10784106553, "process_rss_bytes": 6756958208}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6819935468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total_revenue = 0\n total_cost = 0\n for row in rows:\n if row['revenue'] is not None and row['cost'] is not None:\n total_revenue += row['revenue']\n total_cost += row['cost']\n if total_revenue > 0 and total_cost > 0:\n return total_revenue - total_cost\n else:\n return 0\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1fb443d7753647899bdc487cfa5d58121408e20220200eca14b96b0da3dfe5db", "output_hash": "2c3ad5303d8d1f45396dfca7af62ca9ec49b63f96cdb18c9eca1e0e6dc18ce23"}
91+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "9dd21bff5ab11791cb50d7a5", "attempt_id": "5423139c786b", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:46.848905Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1853, "keep": 463, "compress_ms": 67.42537498939782}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113421, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 112.62058292049915, "doc_prefill_ms": 491.1570419790223, "compress_ms": 67.4783339491114, "serialize_ms": 44.90229196380824, "load_ms": 1.1183340102434158, "receiver_prefill_ms": 0.0, "query_prefill_ms": 42.3420409206301, "ttft_ms": 42.3420409206301, "decode_ms": 11749.81549999211}, "total_wall_ms": 11798.366833943874, "ttft_ms": 42.3420409206301, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 320}, "provider_cached_usage": null, "process_memory": 6758465536, "backend_memory": {"mlx_active_bytes": 6809023496, "mlx_peak_bytes": 6833395092, "mlx_cache_bytes": 13406824505, "process_rss_bytes": 6758465536}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6833395092, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "d9510e09d0ca8eb3b75b9730b22636fac2776b0f1df7ae8a095c1d0f3b1b855b", "output_hash": "96a784ac557a03befea2a2bf3c2c86086644fca8c3acf17e4a7eb949027d72ef"}
92+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e43d7b02322007475f60a2f7", "attempt_id": "c1a0f1b8e31b", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:58.701555Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1853, "keep": 463, "window_tokens": 45, "compress_ms": 248.77916695550084}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113467, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 278.2090000109747, "doc_prefill_ms": 491.1570419790223, "compress_ms": 251.14658300299197, "serialize_ms": 26.764166075736284, "load_ms": 11.841917061246932, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.18933392968029, "ttft_ms": 59.18933392968029, "decode_ms": 2920.7867920631543}, "total_wall_ms": 3273.8528749905527, "ttft_ms": 59.18933392968029, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 79}, "provider_cached_usage": null, "process_memory": 6846595072, "backend_memory": {"mlx_active_bytes": 6781498376, "mlx_peak_bytes": 6819935468, "mlx_cache_bytes": 13700950177, "process_rss_bytes": 6846595072}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6819935468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] != 'Sud':\n if r['units'] >= 2:\n total += r['units'] * (r['revenue'] / 100.0) * 0.09\n return round(total, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "344418aab1ee0e165ccb40fdaea3ae28b2491712429aad0877b2b70e2671f9e4", "output_hash": "8b9932740a653d50bb9a06974b348eac63c0262c73f7ab3adf800acb851a30ab"}
93+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b20154cd293d097bae3a92a0", "attempt_id": "9a6f4f817e57", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:02.238812Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1853, "keep": 463, "window_tokens": 19, "compress_ms": 187.52808298449963}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113467, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 208.3799170795828, "doc_prefill_ms": 491.1570419790223, "compress_ms": 187.91816697921604, "serialize_ms": 20.24979202542454, "load_ms": 2.7195829898118973, "receiver_prefill_ms": 0.0, "query_prefill_ms": 34.48229201603681, "ttft_ms": 34.48229201603681, "decode_ms": 4416.312083951198}, "total_wall_ms": 4461.860541952774, "ttft_ms": 34.48229201603681, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 121}, "provider_cached_usage": null, "process_memory": 6904283136, "backend_memory": {"mlx_active_bytes": 6786085896, "mlx_peak_bytes": 6819935468, "mlx_cache_bytes": 13716859117, "process_rss_bytes": 6904283136}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6819935468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0\n for row in rows:\n if row['region'] != 'Nord' and row['region'] != 'Sud':\n if not row['returned']:\n if row['shipped_days'] < 7:\n total += round((row['revenue'] - row['tax']) * 0.9, 2)\n else:\n total += round((row['revenue'] - row['tax']), 2)\n else:\n total += round(row['revenue'], 2)\n return total", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "344418aab1ee0e165ccb40fdaea3ae28b2491712429aad0877b2b70e2671f9e4", "output_hash": "cf7408abaf3fe928a524f5803f95e73959c0d9bcbfa9c73f0704bc5451e2d084"}
94+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e82238acdb8214dd547b6279", "attempt_id": "bd85d54b6619", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:06.754008Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1853, "keep": 463, "window_tokens": 45, "compress_ms": 133.0997080076486}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113430, "side_information_bytes": 12886, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 158.15033309627324, "doc_prefill_ms": 491.1570419790223, "compress_ms": 134.8713330226019, "serialize_ms": 23.0540829943493, "load_ms": 8.13025003299117, "receiver_prefill_ms": 0.0, "query_prefill_ms": 1112.2218330856413, "ttft_ms": 1112.2218330856413, "decode_ms": 11907.870208960958}, "total_wall_ms": 13189.942457946017, "ttft_ms": 1112.2218330856413, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 320}, "provider_cached_usage": null, "process_memory": 6585827328, "backend_memory": {"mlx_active_bytes": 6809187336, "mlx_peak_bytes": 6850172160, "mlx_cache_bytes": 14307397873, "process_rss_bytes": 6585827328}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6850172160, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] != 'South' and r['region'] != 'South East':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West' and r['region'] != 'South East':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West' and r['region'] != 'South East' and r['region'] != 'South West':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South East':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != '", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "ea03e9d2020e968c315b0fad6087e5c357cc8a44908d6c2fd52e276c88655e8b", "output_hash": "6b997410a720d0cf9c319e5fbf09df9bb348c7fa1c510c5166bedb92e152c749"}
95+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8b9af9604a2dcd74186091cb", "attempt_id": "bbf489b5d01b", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:20.183489Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1853, "keep": 926, "compress_ms": 81.46916702389717, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 56419328, "serialized_payload_bytes": 56446110, "side_information_bytes": 26782, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 181.39470904134214, "doc_prefill_ms": 491.1570419790223, "compress_ms": 127.8945830417797, "serialize_ms": 53.03558299783617, "load_ms": 3.911250038072467, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.55883290152997, "ttft_ms": 59.55883290152997, "decode_ms": 3817.130250041373}, "total_wall_ms": 3883.2200409378856, "ttft_ms": 59.55883290152997, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 96}, "provider_cached_usage": null, "process_memory": 6647808000, "backend_memory": {"mlx_active_bytes": 6786315272, "mlx_peak_bytes": 6845637732, "mlx_cache_bytes": 14337102065, "process_rss_bytes": 6647808000}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6845637732, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] != 'North America':\n continue\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "854e85585cd5598ae69453b12fdffc0e8851d877afbd9e6047416972c1c33dd1", "output_hash": "55874831713d841ac59fbffaa1fb9d32a2ded0513cbc33751bbbec2c460f9881"}
96+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "44c9f24bdb157946f705579d", "attempt_id": "1167560c78f2", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:24.953633Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 82, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.022249994799494743, "decode_payload_ms": 0.0, "receiver_prefill_ms": 64.43300005048513, "ttft_ms": 39.59012497216463, "decode_ms": 90.72749991901219}, "total_wall_ms": 195.6150830956176, "ttft_ms": 39.59012497216463, "input_usage": {"receiver_prompt_tokens": 82}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6661668864, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6510474664, "mlx_cache_bytes": 379634332, "process_rss_bytes": 6661668864}, "peak_prefill_memory": null, "peak_decode_memory": 6510474664, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
97+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "77089f1b85d697b839c288d5", "attempt_id": "0f1eb80e125d", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:25.199886Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 3257, "message_tokens": 3165, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 15217, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013249926269054413, "decode_payload_ms": 0.0, "receiver_prefill_ms": 926.2564160162583, "ttft_ms": 46.42666596919298, "decode_ms": 396.2914999574423}, "total_wall_ms": 1375.4641669802368, "ttft_ms": 46.42666596919298, "input_usage": {"receiver_prompt_tokens": 3257}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6662848512, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7252325440, "mlx_cache_bytes": 1671217820, "process_rss_bytes": 6662848512}, "peak_prefill_memory": null, "peak_decode_memory": 7252325440, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
98+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d69c345c47e7d7fba1448da5", "attempt_id": "3e503b976551", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:26.604964Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 3165}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 613, "message_tokens": 516, "message_utf8_bytes": 2753, "cache_tensor_bytes": null, "serialized_payload_bytes": 2753, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 4.279125016182661, "decode_payload_ms": 0.0, "receiver_prefill_ms": 159.79537495877594, "ttft_ms": 39.40404194872826, "decode_ms": 145.06600005552173}, "total_wall_ms": 346.0776249412447, "ttft_ms": 39.40404194872826, "input_usage": {"receiver_prompt_tokens": 613}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6664994816, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6865649128, "mlx_cache_bytes": 2124532780, "process_rss_bytes": 6664994816}, "peak_prefill_memory": null, "peak_decode_memory": 6865649128, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition approach.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "b4ebdc3f71e0de8c27d74d12c7425f1cc913cf419027bf0aa9d66ba8e1fb4ef4", "output_hash": "c91ced75830c795910278259ba26b352cb3b2bb8bcb4ece33a579acee947dd98"}
99+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "885c9fa0620152363db82124", "attempt_id": "3a29092e34ab", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:26.979472Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 502, "n_sentences": 13}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 595, "message_tokens": 502, "message_utf8_bytes": 2268, "cache_tensor_bytes": null, "serialized_payload_bytes": 2268, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 4.544583964161575, "decode_payload_ms": 0.0, "receiver_prefill_ms": 175.96541601233184, "ttft_ms": 17.405707971192896, "decode_ms": 79.07637499738485}, "total_wall_ms": 274.16608296334743, "ttft_ms": 17.405707971192896, "input_usage": {"receiver_prompt_tokens": 595}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 6665322496, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6855114216, "mlx_cache_bytes": 2478935084, "process_rss_bytes": 6665322496}, "peak_prefill_memory": null, "peak_decode_memory": 6855114216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "179bd6082f22e64b2bb3b7a0fb33d014d40dd0c0a2527b41e66dc1a8cb97a1b6", "output_hash": "65cb98a4671af923622f6d04b76c9ce50b86b153894574e55a962cb4f727b91f"}
100+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e9f57946fcf42b900e80e15d", "attempt_id": "1e532de867fd", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:27.278586Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 500, "n_sentences": 23}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 594, "message_tokens": 501, "message_utf8_bytes": 2495, "cache_tensor_bytes": null, "serialized_payload_bytes": 2495, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.601582975126803, "decode_payload_ms": 0.0, "receiver_prefill_ms": 174.43599994294345, "ttft_ms": 37.115958984941244, "decode_ms": 280.2286660298705}, "total_wall_ms": 494.99341601040214, "ttft_ms": 37.115958984941244, "input_usage": {"receiver_prompt_tokens": 594}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6665338880, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6855114216, "mlx_cache_bytes": 2478935084, "process_rss_bytes": 6665338880}, "peak_prefill_memory": null, "peak_decode_memory": 6855114216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4615384615384615, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "785023803d7f2c38c80debcdc8309ffa1c94e3c99227c52a36e7e66d830d2d9e", "output_hash": "62be2a82b9b81284ad4ad3e5bf7aa56d7866450a753eeed5461895891c88b2e9"}
101+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d65b20dcb306c0140c6cbee5", "attempt_id": "9a679b545f77", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:49.999549Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 609, "message_tokens": 512, "message_utf8_bytes": 2544, "cache_tensor_bytes": null, "serialized_payload_bytes": 2544, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22201.99812494684, "decode_payload_ms": 0.0, "receiver_prefill_ms": 163.27533405274153, "ttft_ms": 33.47712499089539, "decode_ms": 271.6693750116974}, "total_wall_ms": 470.18758300691843, "ttft_ms": 33.47712499089539, "input_usage": {"receiver_prompt_tokens": 609, "sender_input_tokens": 3299, "sender_output_tokens": 512, "sender_prefill_ms": 847.9583340231329, "sender_ttft_ms": 43.58004208188504, "sender_decode_ms": 21307.24516592454}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6666305536, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6863125992, "mlx_cache_bytes": 4080176172, "process_rss_bytes": 6666305536}, "peak_prefill_memory": null, "peak_decode_memory": 6863125992, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4615384615384615, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "a92a3840adade198f1f17e7bef7cf09fa2fc2a50f1fe33c73523c5400c06ace4", "output_hash": "62be2a82b9b81284ad4ad3e5bf7aa56d7866450a753eeed5461895891c88b2e9"}
102+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "6bda13bb6dac6b19c1f79a9b", "attempt_id": "7752cbe610ee", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:12.350377Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 488, "message_tokens": 380, "message_utf8_bytes": 1257, "cache_tensor_bytes": null, "serialized_payload_bytes": 1257, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21855.362500064075, "decode_payload_ms": 0.0, "receiver_prefill_ms": 165.4428340261802, "ttft_ms": 23.285583942197263, "decode_ms": 217.3186249565333}, "total_wall_ms": 407.3195829987526, "ttft_ms": 23.285583942197263, "input_usage": {"receiver_prompt_tokens": 488, "sender_input_tokens": 3329, "sender_output_tokens": 512, "sender_prefill_ms": 949.9937500804663, "sender_ttft_ms": 55.828875047154725, "sender_decode_ms": 20846.271000104025}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6667468800, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6764788824, "mlx_cache_bytes": 4628761644, "process_rss_bytes": 6667468800}, "peak_prefill_memory": null, "peak_decode_memory": 6764788824, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD model", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "8872c1ee69cdc942cca378c38034ceb53ada658177b2b563192861748b58062e", "output_hash": "158b135e4f20814888b9161908fdd8705a04091174ac37b666dd11ae3cc5a4a7"}
103+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "93d51aaa500866412d07fe2f", "attempt_id": "d97b679051fc", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:34.927724Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 858, "message_tokens": 765, "message_utf8_bytes": 830, "cache_tensor_bytes": null, "serialized_payload_bytes": 830, "side_information_bytes": 0, "fetched_bytes": 1660, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22146.301458007656, "decode_payload_ms": 0.0, "receiver_prefill_ms": 227.50466701108962, "ttft_ms": 43.446708004921675, "decode_ms": 36.40404099132866}, "total_wall_ms": 309.42874995525926, "ttft_ms": 43.446708004921675, "input_usage": {"receiver_prompt_tokens": 858, "sender_input_tokens": 3329, "sender_output_tokens": 512, "sender_prefill_ms": 905.3275000769645, "sender_ttft_ms": 68.31662508193403, "sender_decode_ms": 21169.65887497645}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6667829248, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6933646424, "mlx_cache_bytes": 5173365804, "process_rss_bytes": 6667829248}, "peak_prefill_memory": null, "peak_decode_memory": 6933646424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1660, "retrieved_refs": 6}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "93bbce05a459c3fb41371ce4ff355223722554545cc4dac340f74b2cd6065f19", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
104+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c1a6e4b7f5c4a44097a4f3f4", "attempt_id": "9c9d37ae535b", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:35.260676Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 2.8798258894776683}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 3257, "message_tokens": 3165, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 5284, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.15029101632535458, "decode_payload_ms": 0.028749927878379822, "receiver_prefill_ms": 884.1438330709934, "ttft_ms": 55.560875101946294, "decode_ms": 462.33312506228685}, "total_wall_ms": 1407.4600409949198, "ttft_ms": 55.560875101946294, "input_usage": {"receiver_prompt_tokens": 3257}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6668009472, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7252325440, "mlx_cache_bytes": 5173365804, "process_rss_bytes": 6668009472}, "peak_prefill_memory": null, "peak_decode_memory": 7252325440, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
105+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "abd8f72d5ebf8d2a9042c92c", "attempt_id": "f44c40de4bc9", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:37.563174Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 7177860104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 369868800, "serialized_payload_bytes": 369881830, "side_information_bytes": 13030, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 872.6028749952093, "doc_prefill_ms": 768.3387080905959, "compress_ms": 8.387916954234242, "serialize_ms": 90.83100000862032, "load_ms": 7.476166938431561, "receiver_prefill_ms": 0.0, "query_prefill_ms": 65.79979206435382, "ttft_ms": 65.79979206435382, "decode_ms": 491.8956670444459}, "total_wall_ms": 567.7750420290977, "ttft_ms": 65.79979206435382, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 7408992256, "backend_memory": {"mlx_active_bytes": 7553119240, "mlx_peak_bytes": 7646950584, "mlx_cache_bytes": 6101340461, "process_rss_bytes": 7408992256}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7646950584, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "8ee5cf8383bacbbd8af5e34008644b55e21053faeefa305d0b6c29ed3b5977a7", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
106+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "fde7a2e940724dc99dfc4dcd", "attempt_id": "41e64241eecd", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:38.273477Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 7177860104}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "95737b617c283ebf", "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 369868800, "serialized_payload_bytes": 369881858, "side_information_bytes": 13058, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 117.73079202976078, "doc_prefill_ms": 768.3387080905959, "compress_ms": 7.5602500000968575, "serialize_ms": 109.97679200954735, "load_ms": 49.44625007919967, "receiver_prefill_ms": 0.0, "query_prefill_ms": 58.2056250423193, "ttft_ms": 58.2056250423193, "decode_ms": 428.46745892893523}, "total_wall_ms": 538.6375420494005, "ttft_ms": 58.2056250423193, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 7779844096, "backend_memory": {"mlx_active_bytes": 7553119240, "mlx_peak_bytes": 7646950584, "mlx_cache_bytes": 6101340461, "process_rss_bytes": 7779844096}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7646950584, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "06d4d3fe4804f01b770c9587580d9b01847dda9cbf57b892fc812328806ca6c5", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
107+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b9564e291f64162d9ab2f97e", "attempt_id": "d5b9da4adb6f", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:38.939579Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 7005369352}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 196492800, "serialized_payload_bytes": 196520056, "side_information_bytes": 27256, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 103.859500028193, "doc_prefill_ms": 768.3387080905959, "compress_ms": 52.26229201070964, "serialize_ms": 51.22220900375396, "load_ms": 6.104291067458689, "receiver_prefill_ms": 0.0, "query_prefill_ms": 175.92745902948081, "ttft_ms": 175.92745902948081, "decode_ms": 421.5575830312446}, "total_wall_ms": 606.0640841023996, "ttft_ms": 175.92745902948081, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 7977893888, "backend_memory": {"mlx_active_bytes": 7206302728, "mlx_peak_bytes": 7295243472, "mlx_cache_bytes": 7318172845, "process_rss_bytes": 7977893888}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7295243472, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "917cb67928b6971fc0158a2501ad4fe2a4db28dc15dec6a692c9b208d7a1f029", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
108+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2061cab6ef742a32c401c02b", "attempt_id": "e0985f83fa6c", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:39.662899Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6912701448}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 104025600, "serialized_payload_bytes": 104052706, "side_information_bytes": 27106, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 91.90604102332145, "doc_prefill_ms": 768.3387080905959, "compress_ms": 50.19495799206197, "serialize_ms": 41.365165961906314, "load_ms": 4.855667008087039, "receiver_prefill_ms": 0.0, "query_prefill_ms": 145.75187501031905, "ttft_ms": 145.75187501031905, "decode_ms": 435.59229106176645}, "total_wall_ms": 588.821041979827, "ttft_ms": 145.75187501031905, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 8187625472, "backend_memory": {"mlx_active_bytes": 7020049416, "mlx_peak_bytes": 7101129960, "mlx_cache_bytes": 7934092589, "process_rss_bytes": 8187625472}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7101129960, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "089268d279957f99a33245ea777c87a7dfb5186fb242114d885d00081725b7d2", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
109+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "4d3758fd158d22e9937a76e5", "attempt_id": "d588d2ad483d", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:40.340883Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6899856392}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 3225, "keep": 806, "compress_ms": 27.757166884839535}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451421, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 64.23241703305393, "doc_prefill_ms": 768.3387080905959, "compress_ms": 27.796707930974662, "serialize_ms": 36.2354579847306, "load_ms": 1.9377500284463167, "receiver_prefill_ms": 0.0, "query_prefill_ms": 35.60299996752292, "ttft_ms": 35.60299996752292, "decode_ms": 138.23416701052338}, "total_wall_ms": 179.83395792543888, "ttft_ms": 35.60299996752292, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 8184594432, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8174708013, "process_rss_bytes": 8184594432}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition technique", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "cfd45f7f52bfb277d73e198d2a330de3371aa41aec2f77f0731c89ea088e7299", "output_hash": "adc705b949def2e4833f11eabfdb5cc131f5c7801bcaf695f325f05dadc1903f"}
110+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3c1b141113418107b8e9cf68", "attempt_id": "84d644ccfa36", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:40.618881Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6899856392}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 3225, "keep": 806, "compress_ms": 29.710708069615066}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451416, "side_information_bytes": 12888, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 71.83591590728611, "doc_prefill_ms": 768.3387080905959, "compress_ms": 29.746290994808078, "serialize_ms": 41.88670904841274, "load_ms": 3.7346669705584645, "receiver_prefill_ms": 0.0, "query_prefill_ms": 49.082000041380525, "ttft_ms": 49.082000041380525, "decode_ms": 202.58479204494506}, "total_wall_ms": 257.95587501488626, "ttft_ms": 49.082000041380525, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 8278245376, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8199021869, "process_rss_bytes": 8278245376}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Extension of NetVLAD", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4615384615384615, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "bfc5aad5ad47b143ff2ae900af44295c81902b72194b44a72a356052e358fbfb", "output_hash": "8f140fbbbd872bb79f452c7a3e155d08de9d9b3df5f5ad753e80b4846eb8431c"}
111+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ef70aa1aacd6d3a3581298dd", "attempt_id": "c83874a71bb5", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:40.903176Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6807188488}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3225, "keep": 806, "window_tokens": 32, "compress_ms": 166.13045800477266}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451461, "side_information_bytes": 12933, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 196.04720908682793, "doc_prefill_ms": 768.3387080905959, "compress_ms": 168.97325008176267, "serialize_ms": 26.782125001773238, "load_ms": 1.180332968942821, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.43370796740055, "ttft_ms": 59.43370796740055, "decode_ms": 406.5587919903919}, "total_wall_ms": 666.2564580328763, "ttft_ms": 59.43370796740055, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 8287830016, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8623891861, "process_rss_bytes": 8287830016}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "abd1bbad98ec9628f9d26690ae31cf6986bf28f92f185dd733e97b813a4d72f5", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
112+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3ad4f21970c0d4d1b502d40a", "attempt_id": "8d6d9f43c4d7", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:41.752781Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6899856392}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3225, "keep": 806, "window_tokens": 19, "compress_ms": 137.13216700125486}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451461, "side_information_bytes": 12933, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 158.06108398828655, "doc_prefill_ms": 768.3387080905959, "compress_ms": 137.29495892766863, "serialize_ms": 20.584333105944097, "load_ms": 1.8588330131024122, "receiver_prefill_ms": 0.0, "query_prefill_ms": 47.956917085684836, "ttft_ms": 47.956917085684836, "decode_ms": 247.80429096426815}, "total_wall_ms": 300.38979195524007, "ttft_ms": 47.956917085684836, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 8287911936, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8646894997, "process_rss_bytes": 8287911936}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD pooling method.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "abd1bbad98ec9628f9d26690ae31cf6986bf28f92f185dd733e97b813a4d72f5", "output_hash": "bb4768fc890447a743a28d413f4b9165fc5b5887a6b9e7f30a3648c31602aa4e"}
113+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "172f04f7b6a5ab7ec4d24438", "attempt_id": "db4912d84e35", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:42.082088Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6807188488}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 3225, "keep": 806, "window_tokens": 32, "compress_ms": 164.159334031865}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451489, "side_information_bytes": 12961, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 193.5198330320418, "doc_prefill_ms": 768.3387080905959, "compress_ms": 166.54054203536361, "serialize_ms": 26.766416965983808, "load_ms": 11.041625053621829, "receiver_prefill_ms": 0.0, "query_prefill_ms": 1063.2581249810755, "ttft_ms": 1063.2581249810755, "decode_ms": 370.9792080335319}, "total_wall_ms": 1642.524458002299, "ttft_ms": 1063.2581249810755, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7598637056, "backend_memory": {"mlx_active_bytes": 6997996552, "mlx_peak_bytes": 7121915044, "mlx_cache_bytes": 10613161933, "process_rss_bytes": 7598637056}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7121915044, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of the NetVLAD approach.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4285714285714285, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c839577fd550993df17a39b7857d9ea072aa21bed74cdcc84d8e1230292586f1", "output_hash": "35066d4391427300133fbec39fa550f39d33b0cf8700f5300bcb8b4c8f852d68"}
114+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8cddeb96fd95a16ceee43ed2", "attempt_id": "d9a564f246f6", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:43.936211Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6907196424}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 3225, "keep": 1612, "compress_ms": 68.63458303268999, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 98215936, "serialized_payload_bytes": 98243143, "side_information_bytes": 27207, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 183.25416697189212, "doc_prefill_ms": 768.3387080905959, "compress_ms": 120.4705840209499, "serialize_ms": 62.467875075526536, "load_ms": 8.85850004851818, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.84200001694262, "ttft_ms": 59.84200001694262, "decode_ms": 125.22991700097919}, "total_wall_ms": 196.51483302004635, "ttft_ms": 59.84200001694262, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7699415040, "backend_memory": {"mlx_active_bytes": 7009039368, "mlx_peak_bytes": 7073338576, "mlx_cache_bytes": 10619427069, "process_rss_bytes": 7699415040}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7073338576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition technique", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "74c04b99846bb7fc6b88c66a3c6d2a9cba3987192872977c7c90b213d45ae0f0", "output_hash": "adc705b949def2e4833f11eabfdb5cc131f5c7801bcaf695f325f05dadc1903f"}
added runs/raw/E4_smoke.jsonl +60 −0
@@ -0,0 +1,60 @@
1+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e4d6014a524a26cf323d29f6", "attempt_id": "198debfc8cf2", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:05.711305Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 70.56966691743582, "ttft_ms": 13.580082915723324, "decode_ms": 133.09933303389698}, "total_wall_ms": 7606.1354170087725, "ttft_ms": 13.580082915723324, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
2+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3a7862e06d5a28621d6558c3", "attempt_id": "ea464fda49ef", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:05.947390Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 70.54591609630734, "ttft_ms": 13.318874989636242, "decode_ms": 13.681416981853545}, "total_wall_ms": 7486.423374968581, "ttft_ms": 13.318874989636242, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
3+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "8fe0656fc337780a8dd4109b", "attempt_id": "bc2a8af8de0e", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.061895Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 70.04433299880475, "ttft_ms": 13.816624996252358, "decode_ms": 13.376791961491108}, "total_wall_ms": 7486.115916981362, "ttft_ms": 13.816624996252358, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
4+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "d6c66202a9977e9ce9be9b6e", "attempt_id": "df2eb4992ee4", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.175911Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 72.721833945252, "ttft_ms": 13.34204210434109, "decode_ms": 13.484457973390818}, "total_wall_ms": 7488.403292139992, "ttft_ms": 13.34204210434109, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
5+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f9daf53ae4c359f1ffcfe01b", "attempt_id": "e05c9dd05f08", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.292234Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 605, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 80.6656670756638, "ttft_ms": 13.283834094181657, "decode_ms": 119.9960419908166}, "total_wall_ms": 7602.7952920412645, "ttft_ms": 13.283834094181657, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
6+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "5c93c75e9fbdbcaa62c929d5", "attempt_id": "7d85efdf21da", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.521949Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 603, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 83.6761670652777, "ttft_ms": 13.48012499511242, "decode_ms": 133.4958749357611}, "total_wall_ms": 7619.5156670873985, "ttft_ms": 13.48012499511242, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
7+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "0d1c21da0c2ef7ac459880a8", "attempt_id": "3da528f59695", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.768717Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 82.41020806599408, "ttft_ms": 13.700834009796381, "decode_ms": 13.601665967144072}, "total_wall_ms": 7498.5860420856625, "ttft_ms": 13.700834009796381, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "no", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "9390298f3fb0c5b160498935d79cb139aef28e1c47358b4bbba61862b9c26e59"}
8+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "d5baf7618f6c26d9cb7e6e8e", "attempt_id": "b14008ba6e2c", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.895344Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 595, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 87.12095802184194, "ttft_ms": 13.858709018677473, "decode_ms": 13.865667046047747}, "total_wall_ms": 7503.698958083987, "ttft_ms": 13.858709018677473, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
9+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "7a08e0ff0827c646feb5a691", "attempt_id": "c40dbba05788", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.027877Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 611, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 92.70949999336153, "ttft_ms": 13.947332976385951, "decode_ms": 13.76683299895376}, "total_wall_ms": 7509.262875071727, "ttft_ms": 13.947332976385951, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
10+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "715da87439b9c2da1d80846c", "attempt_id": "0288e31dbdee", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.164798Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 603, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 88.89854198787361, "ttft_ms": 13.895749929361045, "decode_ms": 13.664584024809301}, "total_wall_ms": 7505.350458086468, "ttft_ms": 13.895749929361045, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
11+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f670c6c1f9a3016f46e0668a", "attempt_id": "f3db058d1537", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.297840Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 612, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 94.91300000809133, "ttft_ms": 13.864832930266857, "decode_ms": 68.32533294800669}, "total_wall_ms": 7565.9649590961635, "ttft_ms": 13.864832930266857, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
12+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "106680d7384b75e470629b46", "attempt_id": "ff2215fa3b74", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.490963Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 607, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 83.91812501940876, "ttft_ms": 13.685125042684376, "decode_ms": 96.26008395571262}, "total_wall_ms": 7582.891375059262, "ttft_ms": 13.685125042684376, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1379172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "7d9e6c79de4e797c6177e40b49a810a6363041af98982dbd4243baf4bec5c971"}
13+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "69c2449a5a3796644cc816ca", "attempt_id": "730ec68711a2", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.700942Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 602, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 80.60191699769348, "ttft_ms": 13.449500082060695, "decode_ms": 55.076666991226375}, "total_wall_ms": 7538.049542112276, "ttft_ms": 13.449500082060695, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7640", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b7aa5cfd1d2529920befa138d570102a53952edfdd4823a958212feea0698adf"}
14+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "20e3cca49e53c7a7a4486d47", "attempt_id": "be3fa64e72af", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.866998Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 607, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 79.90162493661046, "ttft_ms": 13.807334005832672, "decode_ms": 13.882457977160811}, "total_wall_ms": 7496.488250093535, "ttft_ms": 13.807334005832672, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
15+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "93cddd2a9c80209579d4603d", "attempt_id": "960b65285402", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.989966Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 608, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 80.64408297650516, "ttft_ms": 13.768291915766895, "decode_ms": 13.53445800486952}, "total_wall_ms": 7496.865458087996, "ttft_ms": 13.768291915766895, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
16+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "fed4aa61963a7bf8327a9044", "attempt_id": "e294c1d78b77", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.145297Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 591, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.30716701410711, "ttft_ms": 13.295792043209076, "decode_ms": 133.86049994733185}, "total_wall_ms": 14635.955083067529, "ttft_ms": 13.295792043209076, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
17+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "1fd962b7fed8b7be0f957943", "attempt_id": "a185532acd87", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.379601Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.43875004071742, "ttft_ms": 13.473832979798317, "decode_ms": 13.634915929287672}, "total_wall_ms": 14514.995125005953, "ttft_ms": 13.473832979798317, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
18+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "44736994ac7d3c6e955ca246", "attempt_id": "1ba1eb8cf9e4", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.492830Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.44708398077637, "ttft_ms": 13.740207999944687, "decode_ms": 13.40095791965723}, "total_wall_ms": 14515.074791037478, "ttft_ms": 13.740207999944687, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
19+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3b4059272a0084c78eb7bb71", "attempt_id": "ee0abde30be1", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.605864Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.15895803831518, "ttft_ms": 13.322583981789649, "decode_ms": 13.661875040270388}, "total_wall_ms": 14514.662832953036, "ttft_ms": 13.322583981789649, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
20+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "a5a377217f065df885e17778", "attempt_id": "eab1d937151f", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.718152Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 602, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.07291703484952, "ttft_ms": 13.24529096018523, "decode_ms": 120.1137499883771}, "total_wall_ms": 14621.930750086904, "ttft_ms": 13.24529096018523, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
21+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "75305a0fa99138dc19870aeb", "attempt_id": "1e9b18d736a8", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.937819Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.9936660239473, "ttft_ms": 13.173334067687392, "decode_ms": 133.35729099344462}, "total_wall_ms": 14635.019749985076, "ttft_ms": 13.173334067687392, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
22+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "02cb774895cb7f4200e25ac7", "attempt_id": "f91932e6db80", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.171074Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.08262509386986, "ttft_ms": 13.466249918565154, "decode_ms": 13.275083969347179}, "total_wall_ms": 14514.30133311078, "ttft_ms": 13.466249918565154, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "no", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "9390298f3fb0c5b160498935d79cb139aef28e1c47358b4bbba61862b9c26e59"}
23+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "836a2db482e48950e8112ef4", "attempt_id": "3ec99045954c", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.283280Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 592, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.89300006534904, "ttft_ms": 13.15629098098725, "decode_ms": 13.341167010366917}, "total_wall_ms": 14514.862333075143, "ttft_ms": 13.15629098098725, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
24+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e1d9c69d659fac9dbfa05d73", "attempt_id": "0581ed045141", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.396005Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 608, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.83249995391816, "ttft_ms": 13.473790953867137, "decode_ms": 13.576042023487389}, "total_wall_ms": 14515.38841612637, "ttft_ms": 13.473790953867137, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
25+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "590e8690bc20f0eeb26cf3fe", "attempt_id": "1404cc3d5f3f", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.508754Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.97666589450091, "ttft_ms": 13.296290999278426, "decode_ms": 13.450499973259866}, "total_wall_ms": 14515.232374076732, "ttft_ms": 13.296290999278426, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
26+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "acfb0023ca11a1d053425fb0", "attempt_id": "97c16ca89c34", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.622042Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 609, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.51529199816287, "ttft_ms": 13.40104197151959, "decode_ms": 66.48116698488593}, "total_wall_ms": 14568.905583000742, "ttft_ms": 13.40104197151959, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
27+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3ed2e99bfe065e96ba5567e6", "attempt_id": "1f628ef9ac35", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.788509Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 604, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.03845798317343, "ttft_ms": 13.494165963493288, "decode_ms": 133.29291704576463}, "total_wall_ms": 14635.345748974942, "ttft_ms": 13.494165963493288, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "19,431,432", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "06fe08c35e6644a2505156a3386f0e4d98a25b828e2efd638d491d710f856103"}
28+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f6d8b0874f8a14cbf833d123", "attempt_id": "85a8507f0396", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:17.021966Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 599, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.00924996100366, "ttft_ms": 13.551250100135803, "decode_ms": 52.76383401360363}, "total_wall_ms": 14554.786042077467, "ttft_ms": 13.551250100135803, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7640", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b7aa5cfd1d2529920befa138d570102a53952edfdd4823a958212feea0698adf"}
29+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3bbb5e2647d0d2231c6dc1a3", "attempt_id": "fa24afedd8b2", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:17.174862Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 604, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.16887504141778, "ttft_ms": 13.312499970197678, "decode_ms": 13.794707949273288}, "total_wall_ms": 14515.795915969647, "ttft_ms": 13.312499970197678, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
30+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f81cbc0cf864774a3798a98d", "attempt_id": "40ab50e944d0", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:17.288905Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 605, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 74.4137919973582, "ttft_ms": 13.573125004768372, "decode_ms": 13.354499940760434}, "total_wall_ms": 14519.808249082416, "ttft_ms": 13.573125004768372, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
31+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "ef1cb0065f7c97faa4ef549b", "attempt_id": "4897ddef5806", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.381200Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 426, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 72.67737505026162, "ttft_ms": 13.352832989767194, "decode_ms": 13.35733407177031}, "total_wall_ms": 7289.022208889946, "ttft_ms": 13.352832989767194, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
32+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "5aead85d240368a08066c2c3", "attempt_id": "be42e414d32b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.500359Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 432, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 124.0327499108389, "ttft_ms": 14.377749990671873, "decode_ms": 13.802209054119885}, "total_wall_ms": 7341.845458839089, "ttft_ms": 14.377749990671873, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "55c1fbf7e46378e1c4bdf6ad", "attempt_id": "3bdbc43d86b9", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.670566Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 432, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.62875002902001, "ttft_ms": 13.563584070652723, "decode_ms": 13.158750021830201}, "total_wall_ms": 7269.9129588436335, "ttft_ms": 13.563584070652723, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
34+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "15bba002b57c9391083f8b1a", "attempt_id": "134a07c6c081", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.770854Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 429, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.7230420159176, "ttft_ms": 13.315167045220733, "decode_ms": 13.420167029835284}, "total_wall_ms": 7269.994708942249, "ttft_ms": 13.315167045220733, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
35+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "75dc721488e0ef1126081271", "attempt_id": "bed9a2a1f704", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.869037Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 437, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 74.69737506471574, "ttft_ms": 13.112541986629367, "decode_ms": 144.9349169852212}, "total_wall_ms": 7422.3342089680955, "ttft_ms": 13.112541986629367, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 11}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172 NOK", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "ce5dae2b035af8d38bda9e7d1f1cb7eb7e4141e20298d86d8bce5fe5ac6c0246"}
36+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3e9a22c3f5104a1eddd41703", "attempt_id": "78399ab6c557", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.118864Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 435, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 64.27808292210102, "ttft_ms": 13.261125073768198, "decode_ms": 132.68429203890264}, "total_wall_ms": 7399.776375852525, "ttft_ms": 13.261125073768198, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "BH-71Y-2851", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "38b55d8642902ac72f5987064e67fbe9b766a04dc338df11f6fa45d4cb64179e"}
37+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "eb3ecdabac7df3aa56f2db7d", "attempt_id": "99f7f5ca3819", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.346863Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 429, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.109832922928035, "ttft_ms": 13.361750054173172, "decode_ms": 13.09916703030467}, "total_wall_ms": 7269.102249876596, "ttft_ms": 13.361750054173172, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "no", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "9390298f3fb0c5b160498935d79cb139aef28e1c47358b4bbba61862b9c26e59"}
38+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "1adfc40d070513eb8dcaafa3", "attempt_id": "c00fc0794f37", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.443663Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 427, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 67.15850008185953, "ttft_ms": 13.201624969951808, "decode_ms": 13.237791019491851}, "total_wall_ms": 7283.145666937344, "ttft_ms": 13.201624969951808, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
39+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "119e16c4305375bb8dc1f5c3", "attempt_id": "047930cf5767", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.554627Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 443, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 112.22383310087025, "ttft_ms": 13.389540952630341, "decode_ms": 13.051833026111126}, "total_wall_ms": 7328.250583843328, "ttft_ms": 13.389540952630341, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
40+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "979bd81c65b6db191a8e985d", "attempt_id": "c54b44cc375b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.711472Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 435, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.7681250134483, "ttft_ms": 13.247541966848075, "decode_ms": 13.039291952736676}, "total_wall_ms": 7269.584166933782, "ttft_ms": 13.247541966848075, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
41+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "065a7e3fa74c1ad482af1d3e", "attempt_id": "8fd72b5c73ee", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.809877Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 444, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 110.29041605070233, "ttft_ms": 13.615458039566875, "decode_ms": 13.41954106464982}, "total_wall_ms": 7326.904041925445, "ttft_ms": 13.615458039566875, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
42+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "91f86e20ea07bbdb391d30a6", "attempt_id": "fff9ed3ee7b5", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.964638Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 439, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.64233395084739, "ttft_ms": 13.906834064982831, "decode_ms": 94.21266708523035}, "total_wall_ms": 7351.373499841429, "ttft_ms": 13.906834064982831, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1418934", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b42460a96a2ffc3ec3bc3e8bb110e80b04f38ffa2f4006e5a3159cebdee13b7f"}
43+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "4c81006b8214f8778abfbac8", "attempt_id": "c28310477a2b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:30.144231Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 434, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 68.86416592169553, "ttft_ms": 13.305291067808867, "decode_ms": 13.282250030897558}, "total_wall_ms": 7285.093041951768, "ttft_ms": 13.305291067808867, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
44+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "30262d87fea47fb55f4c9228", "attempt_id": "eea79bc96c46", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:30.258590Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 439, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.78554202616215, "ttft_ms": 13.282875064760447, "decode_ms": 13.299459009431303}, "total_wall_ms": 7269.934750860557, "ttft_ms": 13.282875064760447, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
45+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e2fa4da7d7595a735512203b", "attempt_id": "4e5fc7bd7c56", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:30.357961Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 440, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.51837503258139, "ttft_ms": 13.22641596198082, "decode_ms": 13.591583934612572}, "total_wall_ms": 7269.903499865904, "ttft_ms": 13.22641596198082, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
46+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "8888774dc820faf67464763e", "attempt_id": "bd23292acf1f", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.360272Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 587, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 77.82762497663498, "ttft_ms": 13.771916972473264, "decode_ms": 13.905708095990121}, "total_wall_ms": 15194.374041981064, "ttft_ms": 13.771916972473264, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
47+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "7f668e1f7b3eb3d9b9fd3057", "attempt_id": "e595c673de30", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.483820Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 592, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 74.5352499652654, "ttft_ms": 13.96233297418803, "decode_ms": 13.73537490144372}, "total_wall_ms": 15190.9696678631, "ttft_ms": 13.96233297418803, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
48+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "070822fa247dd66d78b6ffcd", "attempt_id": "6d31e2b2d76b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.603556Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 592, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 73.35308298934251, "ttft_ms": 14.057875028811395, "decode_ms": 13.937833020463586}, "total_wall_ms": 15190.103625995107, "ttft_ms": 14.057875028811395, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
49+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "1c3f890e0a19b625c9227879", "attempt_id": "539634270a2e", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.721629Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 589, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 74.27520793862641, "ttft_ms": 14.271625084802508, "decode_ms": 13.794749975204468}, "total_wall_ms": 15191.149875987321, "ttft_ms": 14.271625084802508, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
50+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "5c48647a9e1b348b9b640926", "attempt_id": "50e5bb1c3127", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.840822Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 598, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 77.12575001642108, "ttft_ms": 14.055041945539415, "decode_ms": 69.60441707633436}, "total_wall_ms": 15249.57975000143, "ttft_ms": 14.055041945539415, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
51+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e6a0f8ca62392e868ec815b1", "attempt_id": "80886cabcadb", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.019147Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 595, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 82.5685829622671, "ttft_ms": 14.223375008441508, "decode_ms": 14.372208039276302}, "total_wall_ms": 15199.900208972394, "ttft_ms": 14.223375008441508, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
52+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "05fe5b3197f86364deed50a4", "attempt_id": "413eadf2083b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.148069Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 589, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 84.24354100134224, "ttft_ms": 14.065457973629236, "decode_ms": 13.865416985936463}, "total_wall_ms": 15200.890749925748, "ttft_ms": 14.065457973629236, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "No", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "1ea442a134b2a184bd5d40104401f2a37fbc09ccf3f4bc9da161c6099be3691d"}
53+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "069d1a413855b163acd583f6", "attempt_id": "5a23a496b605", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.276600Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 587, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 86.33862505666912, "ttft_ms": 14.2137500224635, "decode_ms": 13.89129098970443}, "total_wall_ms": 15203.177750925533, "ttft_ms": 14.2137500224635, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "No", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "1ea442a134b2a184bd5d40104401f2a37fbc09ccf3f4bc9da161c6099be3691d"}
54+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "caeca776a64b5957737b1f15", "attempt_id": "1d487926748b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.407745Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 603, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 91.23012493364513, "ttft_ms": 14.472333015874028, "decode_ms": 13.88770795892924}, "total_wall_ms": 15208.31024996005, "ttft_ms": 14.472333015874028, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
55+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "db28675b1125cfdf5b610b3d", "attempt_id": "0765c60f69f5", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.543531Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 595, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 93.1548330700025, "ttft_ms": 14.234708971343935, "decode_ms": 14.128541923128068}, "total_wall_ms": 15210.25970892515, "ttft_ms": 14.234708971343935, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
56+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "06f0097c3db73ac91f6d5b1f", "attempt_id": "dcb96516ced4", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.681362Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 604, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 90.80570796504617, "ttft_ms": 14.129042043350637, "decode_ms": 14.112583012320101}, "total_wall_ms": 15207.776958937757, "ttft_ms": 14.129042043350637, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
57+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "ba7d480a807e039f9edb854f", "attempt_id": "b01c7d2c7f31", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.816641Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 600, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 89.42750003188848, "ttft_ms": 14.108083094470203, "decode_ms": 68.82829195819795}, "total_wall_ms": 15261.129499878734, "ttft_ms": 14.108083094470203, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7,280,585", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "2b81d1bd559c98218f68f4bc6bf62ababe33d07bcf6ec2cd6c2cd0508c0e52b5"}
58+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "b051902bc46dc13e0a1bf186", "attempt_id": "c6ae50cac079", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:40.005425Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 595, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 87.34066702891141, "ttft_ms": 14.235583948902786, "decode_ms": 13.789458898827434}, "total_wall_ms": 15204.090374871157, "ttft_ms": 14.235583948902786, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
59+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "d011ea48a6559b63fe4420d2", "attempt_id": "183049fa628d", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:40.137750Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 600, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 84.58429202437401, "ttft_ms": 14.115125057287514, "decode_ms": 14.149583061225712}, "total_wall_ms": 15201.539375935681, "ttft_ms": 14.115125057287514, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
60+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "46883052d3de12af358f9963", "attempt_id": "cbe9b3f834c6", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:40.266732Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 601, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 86.56408300157636, "ttft_ms": 13.959750067442656, "decode_ms": 13.88420898001641}, "total_wall_ms": 15203.10266688466, "ttft_ms": 13.959750067442656, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
added runs/raw/E5_smoke.jsonl +80 −0
@@ -0,0 +1,80 @@
1+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "c2531f85c95dbc57153d0f4e", "attempt_id": "6ee0f1aa0a77", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:53.622662Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2417, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 15.056415926665068, "decode_ms": 121.93570903036743, "receiver_prefill_ms": 287.17620798852295}, "total_wall_ms": 434.8010838730261, "ttft_ms": 15.056415926665068, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
2+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "1ed0773614be3aba3c8c6a13", "attempt_id": "58a507aa13dc", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:54.074634Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2415, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.456250075250864, "decode_ms": 134.57829202525318, "receiver_prefill_ms": 263.9893329469487}, "total_wall_ms": 415.5645000282675, "ttft_ms": 13.456250075250864, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
3+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0dd784899f055a526e5323fe", "attempt_id": "e4485ad562f9", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:54.505023Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2423, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.834625016897917, "decode_ms": 95.39945796132088, "receiver_prefill_ms": 272.29566604364663}, "total_wall_ms": 384.97762486804277, "ttft_ms": 13.834625016897917, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7767550", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "87710bb4bb8c5f86714a007462705923c5ce8009326882c4a0f838a7b0005f5c"}
4+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e464b617bf45386c12948550", "attempt_id": "2c77b344b5b6", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:54.904815Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2415, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.825792004354298, "decode_ms": 122.0632919576019, "receiver_prefill_ms": 304.7115419758484}, "total_wall_ms": 444.1995839588344, "ttft_ms": 13.825792004354298, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1,384,522", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "086bd56bc8b1d63bbe425b997da59a545e1bd7ffa6bdd732e726b19220cf2875"}
5+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0f5261a73db9951a26f35b2a", "attempt_id": "967381c85889", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:55.364219Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2424, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 14.14341595955193, "decode_ms": 68.84212500881404, "receiver_prefill_ms": 304.8282921081409}, "total_wall_ms": 391.55841700267047, "ttft_ms": 14.14341595955193, "input_usage": {}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Zenmel-18", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "d324766044a69218e7c53a2be92c812e7edad5aa0821701308fad4d0b3f63e2d"}
6+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "35196571a68a7930a7576c47", "attempt_id": "80b23bba763b", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:55.770805Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2419, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.75274988822639, "decode_ms": 135.45058306772262, "receiver_prefill_ms": 310.9101669397205}, "total_wall_ms": 463.7714169220999, "ttft_ms": 13.75274988822639, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "14,243,807", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "9889c2bdd5bbe834773eb956d4693d3a4e8109c870df394ced69710b96da2b49"}
7+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "84ccc3d49d6380ed8160628a", "attempt_id": "809163a36d9e", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:56.250012Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2419, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.833459001034498, "decode_ms": 13.668250059708953, "receiver_prefill_ms": 295.51949992310256}, "total_wall_ms": 326.49495790246874, "ttft_ms": 13.833459001034498, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
8+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6e8f0133c433a07dfd1271b6", "attempt_id": "ce174c6ef5a7", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:56.591948Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2420, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.64783407188952, "decode_ms": 13.888042070902884, "receiver_prefill_ms": 317.6879999227822}, "total_wall_ms": 348.7097079632804, "ttft_ms": 13.64783407188952, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
9+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6f547971ac924941a7009148", "attempt_id": "2572439e4fb2", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.103232Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1127, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 14.00866697076708, "decode_ms": 93.82133395411074, "receiver_prefill_ms": 121.81266699917614}, "total_wall_ms": 14379.160082899034, "ttft_ms": 14.00866697076708, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6889172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "adcb8d196080006451756ad2fe2106d5a1b6500c0f76ffaa05d7fe704243ad3b"}
10+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e403e9aca77bdb244fd0b4d0", "attempt_id": "d1c277a13a64", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.351018Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1125, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.991833897307515, "decode_ms": 135.3418749058619, "receiver_prefill_ms": 123.2716670492664}, "total_wall_ms": 14421.900999965146, "ttft_ms": 13.991833897307515, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
11+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6baa6b01063b68303a554b17", "attempt_id": "53d6a69496ae", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.641096Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1133, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.811749988235533, "decode_ms": 93.77762500662357, "receiver_prefill_ms": 123.21070895995945}, "total_wall_ms": 14380.13870792929, "ttft_ms": 13.811749988235533, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6068030", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "ce3d061151c7d10d9cc6f1a389ccecd8961b5760dff6c384dfc7a02e8f9222bf"}
12+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "4dd6c587acad2808d5612467", "attempt_id": "93b5e576057f", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.889706Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1125, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.429375016130507, "decode_ms": 120.65325002186, "receiver_prefill_ms": 121.36654101777822}, "total_wall_ms": 14404.811999876983, "ttft_ms": 13.429375016130507, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1,355,485", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "064c9918faa4dc34ddf1b7fd309342033e0c55a27757697cfac056328d0c209e"}
13+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "83d1817abd7e526590a607fc", "attempt_id": "722e64bac111", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.163005Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1134, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.22362502105534, "decode_ms": 67.09000002592802, "receiver_prefill_ms": 121.55058397911489}, "total_wall_ms": 14351.148915942758, "ttft_ms": 13.22362502105534, "input_usage": {}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Zenmel-18", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "d324766044a69218e7c53a2be92c812e7edad5aa0821701308fad4d0b3f63e2d"}
14+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "110a9f2ab259f054abb78608", "attempt_id": "743526c9852a", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.382194Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1129, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.553333934396505, "decode_ms": 134.5761250704527, "receiver_prefill_ms": 120.83662499208003}, "total_wall_ms": 14418.322874000296, "ttft_ms": 13.553333934396505, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "14,203,472", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "1b8d63c935e4628d5567478aa9f1aa1d898d43d55f5be209ba0f208bd190f37a"}
15+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0f99ec00589286ae4ab96da3", "attempt_id": "509282236e10", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.668873Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1129, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.50362494122237, "decode_ms": 13.378208968788385, "receiver_prefill_ms": 121.44350004382432}, "total_wall_ms": 14297.648957930505, "ttft_ms": 13.50362494122237, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
16+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "74f4b20f100fb255543e2545", "attempt_id": "8e0fb07da355", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.834845Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1130, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.468041084706783, "decode_ms": 13.543165987357497, "receiver_prefill_ms": 121.2417499627918}, "total_wall_ms": 14297.607791842893, "ttft_ms": 13.468041084706783, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
17+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ae6de40e70115e38ec1fe95f", "attempt_id": "b27acf424638", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.166118Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 694, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 17.047167057171464, "decode_ms": 120.8459580084309, "receiver_prefill_ms": 77.17541698366404}, "total_wall_ms": 14381.83108298108, "ttft_ms": 17.047167057171464, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
18+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "db1e0004d9835d1cc531df6e", "attempt_id": "3d8a1e931f8d", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.398391Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 692, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.37675005197525, "decode_ms": 133.19487508852035, "receiver_prefill_ms": 76.64595905225724}, "total_wall_ms": 14389.950957032852, "ttft_ms": 13.37675005197525, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
19+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "9a09901f690e45b81ec42ccb", "attempt_id": "279dae943ccf", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.638797Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 700, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 14.127249945886433, "decode_ms": 14.26125003490597, "receiver_prefill_ms": 77.59662496391684}, "total_wall_ms": 14272.744624060579, "ttft_ms": 14.127249945886433, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
20+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "f4cc5af90e025a3438c9375e", "attempt_id": "9f2ae68b8d64", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.761926Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 692, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.230124954134226, "decode_ms": 13.14816705416888, "receiver_prefill_ms": 76.50433294475079}, "total_wall_ms": 14269.677623989992, "ttft_ms": 13.230124954134226, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
21+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "72f5efac4d38ab5dc0654048", "attempt_id": "b72da5e5172f", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.884509Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 701, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.496833038516343, "decode_ms": 66.74774992279708, "receiver_prefill_ms": 76.85845799278468}, "total_wall_ms": 14323.815457057208, "ttft_ms": 13.496833038516343, "input_usage": {}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
22+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "550b173d9301c38240466802", "attempt_id": "94c23122ea16", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:28.058990Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 696, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.47495790105313, "decode_ms": 93.26966700609773, "receiver_prefill_ms": 76.70991704799235}, "total_wall_ms": 14350.178999011405, "ttft_ms": 13.47495790105313, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1415087", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "d8de98946f48b78af60397a05b6e577a8e2416856d227f24711ff800542ffa6a"}
23+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7d1e524827c3ee1a292be484", "attempt_id": "8aa8ea1aba50", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:28.259720Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 696, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.402125099673867, "decode_ms": 13.456832966767251, "receiver_prefill_ms": 76.26687502488494}, "total_wall_ms": 14269.774833112024, "ttft_ms": 13.402125099673867, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
24+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "f35cff3ff329915847b85b7f", "attempt_id": "9af080cdeacc", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:28.380203Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 697, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.48324993159622, "decode_ms": 13.602624996565282, "receiver_prefill_ms": 78.33091600332409}, "total_wall_ms": 14272.0727911219, "ttft_ms": 13.48324993159622, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
25+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "8843c4e0a2c64e04f79d6aa2", "attempt_id": "1496e16b85c7", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:42.643857Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1384, "message_tokens": 1292, "message_utf8_bytes": 3380, "cache_tensor_bytes": null, "serialized_payload_bytes": 3380, "side_information_bytes": 0, "fetched_bytes": 1683, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.578542042523623, "decode_ms": 94.64470902457833, "receiver_prefill_ms": 148.28600001055747}, "total_wall_ms": 14400.122708058916, "ttft_ms": 13.578542042523623, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6889172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "4d600cf3416bdfdbcceef02eedaf880d5773ede73b4ba1dca054e616d4012ee4", "output_hash": "adcb8d196080006451756ad2fe2106d5a1b6500c0f76ffaa05d7fe704243ad3b"}
26+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0f2933272401cc084370967a", "attempt_id": "0e2fc0d8f610", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:42.918501Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1354, "message_tokens": 1264, "message_utf8_bytes": 3303, "cache_tensor_bytes": null, "serialized_payload_bytes": 3303, "side_information_bytes": 0, "fetched_bytes": 1613, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 14.11025004927069, "decode_ms": 134.67420800589025, "receiver_prefill_ms": 144.1560419043526}, "total_wall_ms": 14436.438375036232, "ttft_ms": 14.11025004927069, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "b84ec3106dafb26defcbe7f76d40bd644d33cd7223b6d910a9985a77d2e932b5", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
27+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "89db5fdff414eca1360c5c8b", "attempt_id": "eaef034bb8fe", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.229874Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1415, "message_tokens": 1317, "message_utf8_bytes": 3428, "cache_tensor_bytes": null, "serialized_payload_bytes": 3428, "side_information_bytes": 0, "fetched_bytes": 1727, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.573833974078298, "decode_ms": 13.344207894988358, "receiver_prefill_ms": 152.19508297741413}, "total_wall_ms": 14322.535209008493, "ttft_ms": 13.573833974078298, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "895593c9e120f4063d074d72e0ea36606dc5b4bcacefea4ff70ab64562695bed", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
28+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "cb80418840e30b25cabc7ece", "attempt_id": "7f0458b9139d", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.426967Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1407, "message_tokens": 1317, "message_utf8_bytes": 3428, "cache_tensor_bytes": null, "serialized_payload_bytes": 3428, "side_information_bytes": 0, "fetched_bytes": 1727, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.562500011175871, "decode_ms": 13.312124996446073, "receiver_prefill_ms": 149.3196670198813}, "total_wall_ms": 14319.655666942708, "ttft_ms": 13.562500011175871, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "895593c9e120f4063d074d72e0ea36606dc5b4bcacefea4ff70ab64562695bed", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
29+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7c8760d267881299e3e69f39", "attempt_id": "55271e101d9f", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.620845Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1374, "message_tokens": 1275, "message_utf8_bytes": 3299, "cache_tensor_bytes": null, "serialized_payload_bytes": 3299, "side_information_bytes": 0, "fetched_bytes": 1615, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.59579199925065, "decode_ms": 67.70854198839515, "receiver_prefill_ms": 145.4034170601517}, "total_wall_ms": 14370.145458029583, "ttft_ms": 13.59579199925065, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "a6f42a821d7943c352386b8b4e20945501dc7944212ceca382a93e86f0088dfd", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
30+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "094399c234fd0de4c00855a5", "attempt_id": "bd15bd1e7d12", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.865663Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1411, "message_tokens": 1317, "message_utf8_bytes": 3428, "cache_tensor_bytes": null, "serialized_payload_bytes": 3428, "side_information_bytes": 0, "fetched_bytes": 1727, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.63187504466623, "decode_ms": 134.21929103787988, "receiver_prefill_ms": 151.07270807493478}, "total_wall_ms": 14442.394666024484, "ttft_ms": 13.63187504466623, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "13,751,547", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "7f0d0a79d177c4ebee46fa47933d0ad0511ad6f52d414fd7a3f00743886aa803", "output_hash": "8381b7b8800f64937ec3e0e0d15cdbaf1f2a76bc929b403929b108f2390ad082"}
31+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "40405206bd7a456c61f33574", "attempt_id": "c55532797cd4", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:44.182787Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1378, "message_tokens": 1284, "message_utf8_bytes": 3373, "cache_tensor_bytes": null, "serialized_payload_bytes": 3373, "side_information_bytes": 0, "fetched_bytes": 1673, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.48029205109924, "decode_ms": 13.330582994967699, "receiver_prefill_ms": 148.3127500396222}, "total_wall_ms": 14318.527041003108, "ttft_ms": 13.48029205109924, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "1edfcca5d5af0aeecdf3e6cfce6ea87f864faa495a0061a6b08bd063c3b5e9fa", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
32+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "1eeba3fb9eed49bfded9ff9b", "attempt_id": "c6a2e2770983", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:44.375877Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1392, "message_tokens": 1297, "message_utf8_bytes": 3392, "cache_tensor_bytes": null, "serialized_payload_bytes": 3392, "side_information_bytes": 0, "fetched_bytes": 1691, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.490041019394994, "decode_ms": 13.273874996230006, "receiver_prefill_ms": 149.1100420244038}, "total_wall_ms": 14319.356792024337, "ttft_ms": 13.490041019394994, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "cdee7b9b411dca0cbee7e59f64760f191ceadb11216c0c15f6760e3d5d53486e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "d8bc85caf71ca9f56f3f8d74", "attempt_id": "cd7dca1dd9f6", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:44.842542Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 44, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 63.81474994122982, "decode_ms": 139.43170895799994, "receiver_prefill_ms": 0.0}, "total_wall_ms": 481.60183301661164, "ttft_ms": 63.81474994122982, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n6889172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b9fe7ad55b0c37fb0f5551b72bd99f8cf6f925fd75264f28755ecac8f7652a24"}
34+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "cc10dfa190405328babb888b", "attempt_id": "5b11e161d16d", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.067430Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 42, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.505166061222553, "decode_ms": 182.42304190061986, "receiver_prefill_ms": 0.0}, "total_wall_ms": 485.9277909854427, "ttft_ms": 28.505166061222553, "input_usage": {}, "output_usage": {"output_tokens": 13}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nBH-71Y-2851", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "5551fa62e27e66d44004c639f2de126fbbaadc134a344e5893b9f159865d21a1"}
35+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6bc7337f68d872dfed13b3aa", "attempt_id": "445efebe981c", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.297490Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 50, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.82962499279529, "decode_ms": 140.16608300153166, "receiver_prefill_ms": 0.0}, "total_wall_ms": 443.9334579510614, "ttft_ms": 28.82962499279529, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n7767550", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "3d29dc8364bc9f25a62fdf8895b2508164bb30eff468c6f312e1acc164bfae1d"}
36+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0e411de44dbcd4f72847a3c0", "attempt_id": "0e4e9f4e32b8", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.484772Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 42, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.589040972292423, "decode_ms": 139.41237493418157, "receiver_prefill_ms": 0.0}, "total_wall_ms": 442.85183295141906, "ttft_ms": 28.589040972292423, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n7354655", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "e3a1a67b5dd99a3b78bca513a9fdf83915d81557a4b4dd49abdc6922943cfdb4"}
37+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0b0b601028268214da9df263", "attempt_id": "49dab304374f", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.671251Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 51, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.831708943471313, "decode_ms": 114.26591593772173, "receiver_prefill_ms": 0.0}, "total_wall_ms": 417.0399159193039, "ttft_ms": 28.831708943471313, "input_usage": {}, "output_usage": {"output_tokens": 8}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nTorsel-5", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "98076f21caa823e0185fe09260512bc5a273f6343b0926f76c11ad1fe07fce1e"}
38+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ede55d225cc5358e7f48175d", "attempt_id": "236168bc7697", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.831997Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 46, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.3097920473665, "decode_ms": 181.39916693326086, "receiver_prefill_ms": 0.0}, "total_wall_ms": 484.59966597147286, "ttft_ms": 28.3097920473665, "input_usage": {}, "output_usage": {"output_tokens": 13}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n12,203,327", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "0bfff1a709c4525e7b99586414c4d2d619ea005a7726c0fff6cfea88ba70e322"}
39+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7510675c1038c0e10681915c", "attempt_id": "fa8fcd0ab2bc", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.060697Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 46, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.092040913179517, "decode_ms": 13.7310839490965, "receiver_prefill_ms": 0.0}, "total_wall_ms": 316.7332489974797, "ttft_ms": 28.092040913179517, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
40+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "048dde7c876767f1c7b6ed64", "attempt_id": "101a4e0c2969", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.120774Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 47, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.440083959139884, "decode_ms": 13.840583036653697, "receiver_prefill_ms": 0.0}, "total_wall_ms": 317.2282080631703, "ttft_ms": 28.440083959139884, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
41+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "77c6e32c934f47a56eac7a42", "attempt_id": "689ef527ca9d", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.258781Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2410, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 14.075792045332491, "decode_ms": 96.16091707721353, "receiver_prefill_ms": 265.7509580021724}, "total_wall_ms": 379.7228749608621, "ttft_ms": 14.075792045332491, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1844550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "23de9db5e2c3413d7142a9e77e743ed168ec6de65abf6413935059d107bb8c17"}
42+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "db3b581ff79d7f09f6bf4ab9", "attempt_id": "5b153f170fbe", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.653865Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2413, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.834582990966737, "decode_ms": 137.00370804872364, "receiver_prefill_ms": 265.40816691704094}, "total_wall_ms": 419.9756659800187, "ttft_ms": 13.834582990966737, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "YA-41S-3303", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "2ae4249ea3f767dbc2fc15d5f075c3c9c1ff92b50a630df9d3adb6416607262b"}
43+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ab3ed3250eda6c012c9f752e", "attempt_id": "ecfff760d068", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:47.089921Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2418, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.557915925048292, "decode_ms": 122.66433401964605, "receiver_prefill_ms": 265.6623750226572}, "total_wall_ms": 405.5574160302058, "ttft_ms": 13.557915925048292, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "2,448,885", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "d9cc7ba7227735fd33d7411dda32576c50b6778cb7717cddc1c13d9b861dcc49"}
44+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6f4387c55cf8a5e9242fe62d", "attempt_id": "848e0a7fab23", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:47.511113Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2410, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.478250009939075, "decode_ms": 122.04145907890052, "receiver_prefill_ms": 263.73116695322096}, "total_wall_ms": 402.9229579027742, "ttft_ms": 13.478250009939075, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "2,968,560", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "e16428fe643f862e47e9569d5ec0df386c633fbbeb1a5fdf347a38816d01f1bf"}
45+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ecd0325ac0925265de852893", "attempt_id": "58abafd92f85", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:47.929034Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2420, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.6748329969123, "decode_ms": 80.88966691866517, "receiver_prefill_ms": 264.1993750585243}, "total_wall_ms": 362.57595801725984, "ttft_ms": 13.6748329969123, "input_usage": {}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "d2ace64e89bac774483fbdf2c3b630bf9c232f8f7b2aa06eda5f4a26d74f7cf0"}
46+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "f18f4cfed64113435b36d3ec", "attempt_id": "43eb6195f28e", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:48.307303Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2414, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.73612496536225, "decode_ms": 95.92591703403741, "receiver_prefill_ms": 263.4170410456136}, "total_wall_ms": 376.7490410245955, "ttft_ms": 13.73612496536225, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6447515", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "854fcfa0ae827910332f74fc824602d36fc688e4c310ef595c93498a3e50d16c"}
47+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "adc3d4c18d3e1946ad090c8e", "attempt_id": "f66de66b649f", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:48.699352Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2414, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 14.189749956130981, "decode_ms": 13.982250005938113, "receiver_prefill_ms": 263.9900000067428}, "total_wall_ms": 295.8700000308454, "ttft_ms": 14.189749956130981, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
48+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "bae716a32c678e30f1c7ba76", "attempt_id": "78c87b873f62", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:49.010711Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2416, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.75416701193899, "decode_ms": 13.620915939100087, "receiver_prefill_ms": 264.1928329830989}, "total_wall_ms": 295.36325007211417, "ttft_ms": 13.75416701193899, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
49+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e7141d455cd74ac0f9aaaf24", "attempt_id": "be4d577aba0b", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:03.456810Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1124, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.593790936283767, "decode_ms": 122.23579199053347, "receiver_prefill_ms": 122.03633296303451}, "total_wall_ms": 14394.545415998437, "ttft_ms": 13.593790936283767, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1,844,550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "c3a824f75c26a69ab7e2a0dfde5b194c586a879a105eff635ab136cdb1a4fd6c"}
50+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a32aeec79a9ee8aa0432446f", "attempt_id": "fc67be95d1d7", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:03.732536Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1127, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.491875026375055, "decode_ms": 133.8763750391081, "receiver_prefill_ms": 121.43004103563726}, "total_wall_ms": 14405.404999968596, "ttft_ms": 13.491875026375055, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "PN-55H-5346", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "07e77c9600b8c4977df13983dd328c2c2a77178cdbd678cc76a9cc9784f43993"}
51+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "2c3aab7cdf9d14370dc39bab", "attempt_id": "56d1872e03bc", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.019008Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1132, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.250541989691556, "decode_ms": 93.89745793305337, "receiver_prefill_ms": 121.74612504895777}, "total_wall_ms": 14365.498958039097, "ttft_ms": 13.250541989691556, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3013170", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "a44e28276444065cad5bf760a512ece505a4d855205b9b2bb30a4479693b0a9f"}
52+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7feba5fa129f9153dfaef9f2", "attempt_id": "d6e23cc74256", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.265580Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1124, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.742499984800816, "decode_ms": 122.71799996960908, "receiver_prefill_ms": 121.24983302783221}, "total_wall_ms": 14394.36841593124, "ttft_ms": 13.742499984800816, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "2,968,560", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "e16428fe643f862e47e9569d5ec0df386c633fbbeb1a5fdf347a38816d01f1bf"}
53+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a3d11c9a04eef04742acef19", "attempt_id": "f15db681d709", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.541338Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1134, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.612332986667752, "decode_ms": 82.18583289999515, "receiver_prefill_ms": 123.20812500547618}, "total_wall_ms": 14355.694625061005, "ttft_ms": 13.612332986667752, "input_usage": {}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "d2ace64e89bac774483fbdf2c3b630bf9c232f8f7b2aa06eda5f4a26d74f7cf0"}
54+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "acd92f874c5b04045fba8673", "attempt_id": "57634f0c7ac6", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.777605Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1128, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.54733295738697, "decode_ms": 94.51099997386336, "receiver_prefill_ms": 130.06750005297363}, "total_wall_ms": 14374.922958086245, "ttft_ms": 13.54733295738697, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1449880", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "60b3b6a51e6e9c252943d500d091109937474d1e7422315c117bcba390d4bd51"}
55+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "80c9b8be2a765c02de01ed2b", "attempt_id": "2ff500b209fc", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:05.034415Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1128, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.84162507019937, "decode_ms": 13.439041911624372, "receiver_prefill_ms": 134.0613749343902}, "total_wall_ms": 14298.041917034425, "ttft_ms": 13.84162507019937, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
56+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "024b670d45a38870b1612c72", "attempt_id": "49bcc2b78e8c", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:05.213930Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1130, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.53004202246666, "decode_ms": 13.693084008991718, "receiver_prefill_ms": 140.44683298561722}, "total_wall_ms": 14304.402667097747, "ttft_ms": 13.53004202246666, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
57+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a2db1084287ce9ff31a6b850", "attempt_id": "81949e074967", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:19.578380Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 612, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.298665988259017, "decode_ms": 13.817040948197246, "receiver_prefill_ms": 73.64595902618021}, "total_wall_ms": 14280.350791988894, "ttft_ms": 13.298665988259017, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
58+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7323a45d69829abcf59a311b", "attempt_id": "1665b1c407f6", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:19.696449Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 615, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.854791992343962, "decode_ms": 133.00212495960295, "receiver_prefill_ms": 73.139916989021}, "total_wall_ms": 14399.512876057997, "ttft_ms": 13.854791992343962, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WM-90P-2460", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "3ddcc2bee71a4855b92c73846f02fa6f4653af8f0bfd70329622d6f1c31dccd5"}
59+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "db27f3905a2b89f687d3f767", "attempt_id": "0391283eb189", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:19.933925Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 620, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.558082981035113, "decode_ms": 93.36654201615602, "receiver_prefill_ms": 73.54254205711186}, "total_wall_ms": 14360.009291907772, "ttft_ms": 13.558082981035113, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3013170", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "a44e28276444065cad5bf760a512ece505a4d855205b9b2bb30a4479693b0a9f"}
60+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "3eaa0757a3c73e4332ea1578", "attempt_id": "367d099a0b02", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.131386Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 612, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.241624925285578, "decode_ms": 92.33474999200553, "receiver_prefill_ms": 73.24983295984566}, "total_wall_ms": 14358.351417002268, "ttft_ms": 13.241624925285578, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3039190", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "647471d76579dd2aa6dfa8f84e0a7b0abf733f02c7c2c9fe8239448f2768535e"}
61+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "43d9ce6ef4b176da18ed03cc", "attempt_id": "9c0df373bf58", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.327219Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 622, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.359667034819722, "decode_ms": 13.346542022190988, "receiver_prefill_ms": 73.31933407112956}, "total_wall_ms": 14279.622334055603, "ttft_ms": 13.359667034819722, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
62+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "afff9d86201cec57bd590874", "attempt_id": "bfa5aee9ea74", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.444275Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 616, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.358332915231586, "decode_ms": 92.88116602692753, "receiver_prefill_ms": 72.77325005270541}, "total_wall_ms": 14358.57991699595, "ttft_ms": 13.358332915231586, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6201560", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "0d88693c5d70e48b97fd36a2779621a53e657b2d9bccd2889ca4c8b24c68806d"}
63+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0de15f2f38440a8b76f539aa", "attempt_id": "2b6ed770ca2c", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.640350Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 616, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.727250043302774, "decode_ms": 13.356000068597496, "receiver_prefill_ms": 72.76795792859048}, "total_wall_ms": 14279.413167038001, "ttft_ms": 13.727250043302774, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
64+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "c469452b25b06069f9c119ba", "attempt_id": "804647b655bd", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.757145Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 618, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.705165940336883, "decode_ms": 13.11649999115616, "receiver_prefill_ms": 73.18262499757111}, "total_wall_ms": 14279.700041981414, "ttft_ms": 13.705165940336883, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
65+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7847b741f00b4ce7970c227a", "attempt_id": "9f91842ac6a9", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:43.950387Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1376, "message_tokens": 1287, "message_utf8_bytes": 3340, "cache_tensor_bytes": null, "serialized_payload_bytes": 3340, "side_information_bytes": 0, "fetched_bytes": 1871, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 41.1847080104053, "decode_ms": 282.48854097910225, "receiver_prefill_ms": 331.38612494803965}, "total_wall_ms": 23733.73899993021, "ttft_ms": 41.1847080104053, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1844550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ae95d76b4a91d3570273b15d8c20d44f5b348cc24d5dfdf7ae217730c17e2ab0", "output_hash": "23de9db5e2c3413d7142a9e77e743ed168ec6de65abf6413935059d107bb8c17"}
66+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "18d629a7d19c493521afdd59", "attempt_id": "40723f42711e", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:44.633980Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1338, "message_tokens": 1246, "message_utf8_bytes": 3244, "cache_tensor_bytes": null, "serialized_payload_bytes": 3244, "side_information_bytes": 0, "fetched_bytes": 1771, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 42.46612498536706, "decode_ms": 386.4938340848312, "receiver_prefill_ms": 330.83112502936274}, "total_wall_ms": 23837.171457940713, "ttft_ms": 42.46612498536706, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WM-90P-2460", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "c903e22acc833959a3bbf0638fb1ee9fae27616194b965775737c0b7672960fb", "output_hash": "3ddcc2bee71a4855b92c73846f02fa6f4653af8f0bfd70329622d6f1c31dccd5"}
67+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "4ced35996da610c099012a59", "attempt_id": "820eb88fbc9e", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:45.416955Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1365, "message_tokens": 1268, "message_utf8_bytes": 3301, "cache_tensor_bytes": null, "serialized_payload_bytes": 3301, "side_information_bytes": 0, "fetched_bytes": 1832, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 19.3950830725953, "decode_ms": 337.39804197102785, "receiver_prefill_ms": 286.93091694731265}, "total_wall_ms": 23720.98304098472, "ttft_ms": 19.3950830725953, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3013170", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "fe2ea5528397251033d4174a5ce2849e153ab545799d9cb0d18c19e1f74399a5", "output_hash": "a44e28276444065cad5bf760a512ece505a4d855205b9b2bb30a4479693b0a9f"}
68+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a9a8a0c4d432bb5ca8865856", "attempt_id": "0a718d636e3a", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:46.084217Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1355, "message_tokens": 1266, "message_utf8_bytes": 3293, "cache_tensor_bytes": null, "serialized_payload_bytes": 3293, "side_information_bytes": 0, "fetched_bytes": 1828, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 27.289082994684577, "decode_ms": 185.85829203948379, "receiver_prefill_ms": 375.7223749998957}, "total_wall_ms": 23666.18524992373, "ttft_ms": 27.289082994684577, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3039190", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "c6e87a60fb8104d185066ed693f734b701055dbd16ad4805501b55fce4a1c26a", "output_hash": "647471d76579dd2aa6dfa8f84e0a7b0abf733f02c7c2c9fe8239448f2768535e"}
69+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "5f42863ad361430b395ee32b", "attempt_id": "476d4bfd78d0", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:46.697210Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1374, "message_tokens": 1275, "message_utf8_bytes": 3285, "cache_tensor_bytes": null, "serialized_payload_bytes": 3285, "side_information_bytes": 0, "fetched_bytes": 1827, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 38.89429196715355, "decode_ms": 223.2152500655502, "receiver_prefill_ms": 352.5512919295579}, "total_wall_ms": 23691.884166910313, "ttft_ms": 38.89429196715355, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "d49bf0d3d8d9b187ec63d83e81a8c5576985a6752461bd3bad41868126698f4e", "output_hash": "d2ace64e89bac774483fbdf2c3b630bf9c232f8f7b2aa06eda5f4a26d74f7cf0"}
70+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e63b5a7f8f10a8640c22b3d3", "attempt_id": "4a99b3d331ad", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:47.335961Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1380, "message_tokens": 1287, "message_utf8_bytes": 3340, "cache_tensor_bytes": null, "serialized_payload_bytes": 3340, "side_information_bytes": 0, "fetched_bytes": 1871, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 20.951000042259693, "decode_ms": 286.4009999902919, "receiver_prefill_ms": 358.3094170317054}, "total_wall_ms": 23743.166374973953, "ttft_ms": 20.951000042259693, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "4403065", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ae95d76b4a91d3570273b15d8c20d44f5b348cc24d5dfdf7ae217730c17e2ab0", "output_hash": "b06841909baef4e457116b1a8639e2707ad180e88be6a26a03e41f03f77fbb2b"}
71+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "920a8167c2783d722d0f0f92", "attempt_id": "33f897294bd5", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:48.023335Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1339, "message_tokens": 1246, "message_utf8_bytes": 3244, "cache_tensor_bytes": null, "serialized_payload_bytes": 3244, "side_information_bytes": 0, "fetched_bytes": 1771, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 42.38762496970594, "decode_ms": 35.475166980177164, "receiver_prefill_ms": 324.8105419334024}, "total_wall_ms": 23479.923208011314, "ttft_ms": 42.38762496970594, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "c903e22acc833959a3bbf0638fb1ee9fae27616194b965775737c0b7672960fb", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
72+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "c0005b4a22a9363b2616fef6", "attempt_id": "1181aba337a9", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:48.447224Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1350, "message_tokens": 1255, "message_utf8_bytes": 3257, "cache_tensor_bytes": null, "serialized_payload_bytes": 3257, "side_information_bytes": 0, "fetched_bytes": 1792, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 37.491583032533526, "decode_ms": 48.52670803666115, "receiver_prefill_ms": 243.10816603247076}, "total_wall_ms": 23406.509208027273, "ttft_ms": 37.491583032533526, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9b59bd103f78834d60cc7217f03f102d22542a7ee645009dcdac687299160ac4", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
73+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "07f6e52a778e66864527904a", "attempt_id": "b43576eb2684", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:49.502726Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 41, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 98.5616659745574, "decode_ms": 590.0328749557957, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1401.089790975675, "ttft_ms": 98.5616659745574, "input_usage": {}, "output_usage": {"output_tokens": 16}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n1844550\n<|im_start|>robatni\nunknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 0.4}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "26ce51c96d59175d93d6596952c2cf98af15270047cff7dd5d0817be6651daf6"}
74+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "825a15b690efcce18cae054b", "attempt_id": "6c59b49831bb", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:50.221140Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 44, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 63.614208018407226, "decode_ms": 432.6625830726698, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1206.9736659759656, "ttft_ms": 63.614208018407226, "input_usage": {}, "output_usage": {"output_tokens": 13}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nYA-41S-3303", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "7063a3151528c78281542b673393a8cdd101b2d06b1e8adb0f19815928a53354"}
75+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "277cb9d37e93d596669dfb7d", "attempt_id": "9d1ef8ed7d31", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:50.743330Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 49, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 58.28074994497001, "decode_ms": 481.99387500062585, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1248.6895410111174, "ttft_ms": 58.28074994497001, "input_usage": {}, "output_usage": {"output_tokens": 12}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n1,844,550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "c68142a24d688d117833ba79e64a8b34758a2d53fe8bc5337bc0d389dcbc80b5"}
76+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "5e9479fcaa84ed4182367340", "attempt_id": "e0c3d54d964e", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:51.305504Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 41, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 49.72116707358509, "decode_ms": 365.5605419771746, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1123.2687489828095, "ttft_ms": 49.72116707358509, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3,039,190", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "f540ce7783d75dedddec953cb68a2b0f2472cd0a59b02bdb333d9cb537732b77"}
77+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "3e2f86abebf899aa1757af38", "attempt_id": "e67e1fe80273", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:51.743468Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 51, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 90.47212498262525, "decode_ms": 318.9096669666469, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1122.882082941942, "ttft_ms": 90.47212498262525, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nLithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "92ab911faec640c58ff3f6e7d4d454de4d628c70c92cc85dd752f841f6aea1a6"}
78+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "74fb9e5eba23585fa3581ae4", "attempt_id": "d6ea887eb292", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:52.181156Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 45, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 55.63445796724409, "decode_ms": 437.1588749345392, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1200.9254159638658, "ttft_ms": 55.63445796724409, "input_usage": {}, "output_usage": {"output_tokens": 11}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n14290.42", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "c7569b7c33261a61449230bc152f178f7cc210ce3800ad959c0d4d3b98e4a541"}
79+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "b46f68e2943921254526a4cd", "attempt_id": "4a0e1f421332", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:52.697221Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 45, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 40.74966604821384, "decode_ms": 24.908291990868747, "receiver_prefill_ms": 0.0}, "total_wall_ms": 774.1670409450307, "ttft_ms": 40.74966604821384, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
80+{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "36c71d3273f01bddf02acad4", "attempt_id": "8c697cd8267f", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:52.786717Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 47, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 68.34420806262642, "decode_ms": 22.9576249839738, "receiver_prefill_ms": 0.0}, "total_wall_ms": 800.9481240296736, "ttft_ms": 68.34420806262642, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
added runs/raw/chains/E4_smoke_syn_02000_2000_A4_closed_qwen3b.json +74 −0
@@ -0,0 +1,74 @@
1+{
2+ "hops": {
3+ "1": "**Summary:**\n\n- **Capital Projects Register, Issue 165**\n - **Proposals under review:** Zennovvol-7, Ruktor-14, Torsel-10 (no attributes recorded)\n - **Project Zennovvol-6:**\n - Proposed by Idris Abubakar on 5 Nov 2033\n - Initial budget: 6,889,172 NOK\n - Revised budget: 7,280,585 NOK\n - Approved on 8 Mar 2034\n - Site: Lot 208-C, Senneterre\n - Supplier: BH-71Y-2851\n - Launch postponed to 22 Apr 2034\n - Rated capacity: 35.2 km\n - Calibration constant: 58.0820\n - **Project Ruktor-10:**\n - Proposed by Teodora Mäkelä on 2 Jul 2032\n - Initial budget: 5,869,675 NOK\n - Revised budget: 5,802,310 NOK\n - Approved on 27 Oct 2032\n - Site: Lot 50-D, Matagami\n - Supplier: SP-38G-4217\n - Cancelled\n - Rated capacity: 30.99 MW\n - Calibration constant: 54.0816\n - **Project Torsel-5:**\n - Proposed by Haruki Rousseau on 18 Apr 2031\n - Initial budget: 1,661,075 CAD\n - Revised budget: 1,945,685 CAD\n - Approved on 13 Sep 2031\n - Site: Lot 835-P, Fermont\n - Supplier: WG-14H-1182\n - On schedule\n - Rated capacity: 7.64 MW\n - Calibration constant: 68.6281\n - **Project Eshtal-12:**\n - Proposed by Oksana Okonkwo on 23",
4+ "2": "**Summary:**\n- **Capital Projects Register, Issue 165**\n - **Proposals under review:** Zennovvol-7, Ruktor-14, Torsel-10 (no attributes recorded)\n - **Project Zennovvol-6:**\n - Proposed by Idris Abubakar on 5 Nov 2033\n - Initial budget: 6,889,172 NOK\n - Revised budget: 7,280,585 NOK\n - Approved on 8 Mar 2034\n - Site: Lot 208-C, Senneterre\n - Supplier: BH-71Y-2851\n - Launch postponed to 22 Apr 2034\n - Rated capacity: 35.2 km\n - Calibration constant: 58.0820\n - **Project Ruktor-10:**\n - Proposed by Teodora Mäkelä on 2 Jul 2032\n - Initial budget: 5,869,675 NOK\n - Revised budget: 5,802,310 NOK\n - Approved on 27 Oct 2032\n - Site: Lot 50-D, Matagami\n - Supplier: SP-38G-4217\n - Cancelled\n - Rated capacity: 30.99 MW\n - Calibration constant: 54.0816\n - **Project Torsel-5:**\n - Proposed by Haruki Rousseau on 18 Apr 2031\n - Initial budget: 1,661,075 CAD\n - Revised budget: 1,945,685 CAD\n - Approved on 13 Sep 2031\n - Site: Lot 835-P, Fermont\n - Supplier: WG-14H-1182\n - On schedule\n - Rated capacity: 7.64 MW\n - Calibration constant: 68.6281\n - **Project Eshtal-12:** Proposed by Oksana Okonkwo on 23"
5+ },
6+ "meta": {
7+ "1": {
8+ "hop_ms": 7388.310500071384,
9+ "agent": "mlx-community/Qwen2.5-3B-Instruct-bf16",
10+ "usage": {
11+ "sender_input_tokens": 2087,
12+ "sender_output_tokens": 512,
13+ "sender_prefill_ms": 258.550624945201,
14+ "sender_ttft_ms": 14.011999941430986,
15+ "sender_decode_ms": 7108.514915918931
16+ },
17+ "repair": {
18+ "lookups": 0,
19+ "fetched_bytes": 0
20+ },
21+ "ledger": {
22+ "facts_total": 64,
23+ "facts_retained": 28,
24+ "retention_rate": 0.4375,
25+ "retention_by_kind": {
26+ "person": 0.6666666666666666,
27+ "date": 0.0,
28+ "amount": 0.42857142857142855,
29+ "status": 0.75,
30+ "place": 0.0,
31+ "identifier": 0.5,
32+ "quantity": 0.5
33+ },
34+ "unsupported_numbers": 0,
35+ "unsupported_identifiers": 0,
36+ "distorted_sentences": 0,
37+ "message_bytes": 1196
38+ }
39+ },
40+ "2": {
41+ "hop_ms": 7029.634707956575,
42+ "agent": "mlx-community/Qwen2.5-3B-Instruct-bf16",
43+ "usage": {
44+ "sender_input_tokens": 629,
45+ "sender_output_tokens": 509,
46+ "sender_prefill_ms": 86.81391598656774,
47+ "sender_ttft_ms": 13.69220798369497,
48+ "sender_decode_ms": 6928.4298330312595
49+ },
50+ "repair": {
51+ "lookups": 0,
52+ "fetched_bytes": 0
53+ },
54+ "ledger": {
55+ "facts_total": 64,
56+ "facts_retained": 28,
57+ "retention_rate": 0.4375,
58+ "retention_by_kind": {
59+ "person": 0.6666666666666666,
60+ "date": 0.0,
61+ "amount": 0.42857142857142855,
62+ "status": 0.75,
63+ "place": 0.0,
64+ "identifier": 0.5,
65+ "quantity": 0.5
66+ },
67+ "unsupported_numbers": 0,
68+ "unsupported_identifiers": 0,
69+ "distorted_sentences": 0,
70+ "message_bytes": 1189
71+ }
72+ }
73+ }
74+}
\ No newline at end of file
added runs/raw/chains/E4_smoke_syn_02000_2000_A5_repairable_qwen3b_llama3b.json +75 −0
@@ -0,0 +1,75 @@
1+{
2+ "hops": {
3+ "1": "{\"facts\": [{\"entity\": \"Zennovvol-6\", \"attribute\": \"proposed by\", \"value\": \"Idris Abubakar\", \"span\": [238, 321]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"initial budget\", \"value\": \"6,889,172 NOK\", \"span\": [323, 433]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"final budget\", \"value\": \"7,280,585 NOK\", \"span\": [434, 508]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"approval date\", \"value\": \"8 March 2034\", \"span\": [509, 576]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"launch date\", \"value\": \"22 April 2034\", \"span\": null}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"rated capacity\", \"value\": \"35.2 km\", \"span\": [759, 804]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"supplier identifier\", \"value\": \"BH-71Y-2851\", \"span\": [629, 684]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"status\", \"value\": \"NOT cancelled\", \"span\": [685, 758]}]}",
4+ "2": "{\"facts\": [{\"entity\": \"Zennovvol-6\", \"attribute\": \"proposed by\", \"value\": \"Idris Abubakar\", \"quote\": \"Project Zennovvol-6 was proposed by Idris Abubakar on 5 November 2033 with an initial budget of 6,889,172 NOK.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"initial budget\", \"value\": \"6,889,172 NOK\", \"quote\": \"Project Zennovvol-6 was proposed by Idris Abubakar on 5 November 2033 with an initial budget of 6,889,172 NOK.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"final budget\", \"value\": \"7,280,585 NOK\", \"quote\": \"On 19 January 2034 the budget of Zennovvol-6 was revised to 7,280,585 NOK.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"approval date\", \"value\": \"8 March 2034\", \"quote\": \"The approved decision for Zennovvol-6 was recorded on 8 March 2034.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"launch date\", \"value\": \"22 April 2034\", \"quote\": \"Zennovvol-6 was NOT cancelled; its launch was postponed to 22 April 2034.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"rated capacity\", \"value\": \"35.2 km\", \"quote\": \"The rated capacity of Zennovvol-6 is 35.2 km.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"supplier identifier\", \"value\": \"BH-71Y-2851\", \"quote\": \"The supplier identifier for Zennovvol-6 is BH-71Y-2851.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"status\", \"value\": \"NOT cancelled\", \"quote\": \"Zennovvol-6 was NOT cancelled; its launch was postponed to 22 April 2034.\"}], \"notes\": \"\", \"unknowns\": []}"
5+ },
6+ "meta": {
7+ "1": {
8+ "hop_ms": 7189.094333909452,
9+ "agent": "mlx-community/Qwen2.5-3B-Instruct-bf16",
10+ "usage": {
11+ "sender_input_tokens": 2115,
12+ "sender_output_tokens": 512,
13+ "sender_prefill_ms": 247.3138750065118,
14+ "sender_ttft_ms": 13.779832981526852,
15+ "sender_decode_ms": 6926.056583062746
16+ },
17+ "repair": {
18+ "lookups": 0,
19+ "fetched_bytes": 0
20+ },
21+ "ledger": {
22+ "facts_total": 64,
23+ "facts_retained": 10,
24+ "retention_rate": 0.15625,
25+ "retention_by_kind": {
26+ "person": 0.16666666666666666,
27+ "date": 0.125,
28+ "amount": 0.14285714285714285,
29+ "status": 0.3333333333333333,
30+ "place": 0.0,
31+ "identifier": 0.08333333333333333,
32+ "quantity": 0.16666666666666666
33+ },
34+ "unsupported_numbers": 0,
35+ "unsupported_identifiers": 0,
36+ "distorted_sentences": 0,
37+ "message_bytes": 817
38+ }
39+ },
40+ "2": {
41+ "hop_ms": 7899.107125005685,
42+ "agent": "mlx-community/Llama-3.2-3B-Instruct-bf16",
43+ "usage": {
44+ "sender_input_tokens": 709,
45+ "sender_output_tokens": 492,
46+ "sender_prefill_ms": 825.0708330888301,
47+ "sender_ttft_ms": 37.18812495935708,
48+ "sender_decode_ms": 7024.092499981634
49+ },
50+ "repair": {
51+ "lookups": 1,
52+ "fetched_bytes": 540,
53+ "codes_mentioned": 1
54+ },
55+ "ledger": {
56+ "facts_total": 64,
57+ "facts_retained": 16,
58+ "retention_rate": 0.25,
59+ "retention_by_kind": {
60+ "person": 0.16666666666666666,
61+ "date": 0.25,
62+ "amount": 0.14285714285714285,
63+ "status": 0.75,
64+ "place": 0.0,
65+ "identifier": 0.08333333333333333,
66+ "quantity": 0.16666666666666666
67+ },
68+ "unsupported_numbers": 0,
69+ "unsupported_identifiers": 0,
70+ "distorted_sentences": 0,
71+ "message_bytes": 1403
72+ }
73+ }
74+ }
75+}
\ No newline at end of file
added scripts/e6_network.py +76 −0
@@ -0,0 +1,76 @@
1+"""E6/E2: genuine two-node cache transfer over the LAN (laptop -> M3U96b worker) vs text transfer vs local recompute.
2+Usage: python scripts/e6_network.py --host 192.168.2.85 --port 9430 --model qwen3b --docs 6 --lengths 2000 8000 16000 --repeats 3
3+Requires the worker: on the remote node `python -m wp30 worker --port 9430 --model <repo>` (same checkpoint revision).
4+Loopback mode (--host 127.0.0.1) is labelled 'loopback' and is NOT a network measurement."""
5+import argparse, json, time, statistics, subprocess, sys, socket
6+import yaml, numpy as np, mlx.core as mx
7+from wp30.util import ROOT, read_json, utc_now, append_jsonl, sha256_text
8+from wp30.data.prepare import load_doc, load_key
9+from wp30.backends.mlx_backend import MLXModel
10+from wp30.cache.sparse import quantize_cache, cache_tensor_bytes, to_sparse_all
11+from wp30.cache.serialize import build_manifest, serialize_cache, codec_compress
12+from wp30.transport import Client
13+from wp30.prompts import SYSTEM_RECEIVER, receiver_user_text
14+from wp30.scoring import score_synthetic
15+
16+ap = argparse.ArgumentParser()
17+ap.add_argument("--host", required=True); ap.add_argument("--port", type=int, default=9430); ap.add_argument("--model", default="qwen3b")
18+ap.add_argument("--docs", type=int, default=6); ap.add_argument("--lengths", type=int, nargs="+", default=[2000, 8000, 16000]); ap.add_argument("--repeats", type=int, default=3)
19+ap.add_argument("--label", default=None); ap.add_argument("--split", default="dev"); ap.add_argument("--out", default="runs/raw/E6_network.jsonl")
20+a = ap.parse_args()
21+label = a.label or ("loopback" if a.host.startswith("127.") else "lan")
22+mcfg = yaml.safe_load(open(ROOT / "configs/models.yaml"))["models"]
23+m = MLXModel(mcfg[a.model]["repo"])
24+cl = Client(a.host, a.port)
25+h, _, meta = cl.call({"cmd": "ping"})
26+assert h["config_hash"] == m.info.config_hash, "remote model differs"
27+print("remote ok", h["model_revision"], "rtt_ms", round(meta["roundtrip_ms"], 2))
28+# raw link bandwidth probe with an incompressible 64 MiB payload (frame echo is not implemented; use run_text no-op? -> use a dedicated ping with payload)
29+man = read_json(ROOT / "data/manifests/splits.json")
30+items = [it for it in man["items"] if it["family"] == "synthetic" and it["split"] == a.split and it.get("target_tokens") in a.lengths]
31+by_len = {L: [it for it in items if it["target_tokens"] == L][: a.docs] for L in a.lengths}
32+out = ROOT / a.out
33+for L, its in by_len.items():
34+ for it in its:
35+ doc = load_doc(it["document_id"], "synthetic"); key = load_key(it["document_id"], "synthetic")
36+ q = doc["questions"][0]; qk = key["questions"][0]
37+ full = m.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text("Document", doc["text"], q["question"], "synthetic")}])
38+ pre_txt = m.tokenizer.apply_chat_template([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": "Document:\n" + doc["text"] + "\n\n"}], add_generation_prompt=False, tokenize=False)
39+ pre = m.encode(pre_txt); n = 0
40+ while n < min(len(pre), len(full)) and pre[n] == full[n]: n += 1
41+ pre, query = full[:n], full[n:]
42+ for rep in range(a.repeats):
43+ # (1) local sender prefill + serialize + transfer cache + remote load + remote query [warm remote cache scenario: cache moved]
44+ for variant in [("B1", 8 if False else None, "none"), ("B1", None, "zstd"), ("B2", 8, "none"), ("B2", 4, "none")]:
45+ meth, bits, codec = variant
46+ t0 = time.perf_counter(); c, tp = m.prefill(pre)
47+ c2 = to_sparse_all(c) if bits is None else quantize_cache(c, bits)
48+ mx.eval([x for cc in c2 for x in (cc.keys if isinstance(cc.keys, tuple) else (cc.keys,))]); mx.synchronize()
49+ t1 = time.perf_counter()
50+ manf = build_manifest(m.info, pre, c2, it["document_id"], meth, {"bits": bits})
51+ payload = serialize_cache(c2, manf); t2 = time.perf_counter()
52+ comp, cms = codec_compress(payload, codec) if codec != "none" else (payload, 0.0)
53+ hdr = {"cmd": "run_cache", "codec": codec, "prefix_hash": manf["prefix_token_hash"], "query_ids": query, "max_tokens": 32}
54+ rh, _, tm = cl.call(hdr, comp)
55+ rec = {"experiment_id": "E6_network", "label": label, "timestamp_utc": utc_now(), "document_id": it["document_id"], "target_tokens": L, "prefix_tokens": len(pre), "query_tokens": len(query), "repeat": rep,
56+ "method": meth, "bits": bits, "codec": codec, "status": rh.get("status"), "answer": rh.get("text"), "correct": score_synthetic(rh.get("text", ""), qk)["correct"] if rh.get("text") else None,
57+ "cache_tensor_bytes": cache_tensor_bytes(c2), "serialized_bytes": len(payload), "transport_bytes": tm["transport_bytes"], "sender_prefill_ms": tp["prefill_ms"], "compress_ms": (t1 - t0) * 1000 - tp["prefill_ms"],
58+ "serialize_ms": (t2 - t1) * 1000, "codec_ms": cms, "send_ms": tm["send_ms"], "roundtrip_ms": tm["roundtrip_ms"], "remote_recv_ms": rh.get("recv_ms"), "remote_decompress_ms": rh.get("decompress_ms"),
59+ "remote_load_ms": rh.get("load_ms"), "remote_ttft_ms": rh.get("ttft_ms"), "remote_decode_ms": rh.get("decode_ms"), "remote_server_ms": rh.get("server_ms"), "remote_peak_bytes": rh.get("peak_bytes"),
60+ "effective_gbps": (tm["transport_bytes"] * 8 / 1e9) / max(1e-9, (tm["roundtrip_ms"] - (rh.get("server_ms") or 0)) / 1000)}
61+ append_jsonl(out, rec)
62+ print(f"{label} L={L} {meth} bits={bits} codec={codec} rep={rep} bytes={len(comp):>10d} send={tm['send_ms']:7.1f}ms rt={tm['roundtrip_ms']:7.1f}ms remote_load={rh.get('load_ms',0):6.1f} ttft={rh.get('ttft_ms',0):6.1f} eff={rec['effective_gbps']:.2f}Gb/s corr={rec['correct']}")
63+ # (2) text transfer: send token ids, remote does full prefill [cold remote scenario]
64+ rh, _, tm = cl.call({"cmd": "run_text", "token_ids": full, "max_tokens": 32})
65+ rec = {"experiment_id": "E6_network", "label": label, "timestamp_utc": utc_now(), "document_id": it["document_id"], "target_tokens": L, "prefix_tokens": len(pre), "query_tokens": len(query), "repeat": rep,
66+ "method": "text_remote_prefill", "bits": None, "codec": "none", "status": "success", "answer": rh.get("text"), "correct": score_synthetic(rh.get("text", ""), qk)["correct"],
67+ "serialized_bytes": len(doc["text"].encode()), "transport_bytes": tm["transport_bytes"], "send_ms": tm["send_ms"], "roundtrip_ms": tm["roundtrip_ms"], "remote_prefill_ms": rh.get("prefill_ms"),
68+ "remote_ttft_ms": rh.get("ttft_ms"), "remote_decode_ms": rh.get("decode_ms"), "remote_server_ms": rh.get("server_ms"), "remote_peak_bytes": rh.get("peak_bytes")}
69+ append_jsonl(out, rec)
70+ print(f"{label} L={L} text_remote rep={rep} bytes={tm['transport_bytes']:>10d} rt={tm['roundtrip_ms']:7.1f}ms remote_prefill={rh.get('prefill_ms',0):7.1f} ttft={rh.get('ttft_ms',0):6.1f}")
71+ # (3) local processing (no transfer): run where the data is
72+ c, tp = m.prefill(pre); g = m.generate_from_cache(c, query, max_tokens=32)
73+ append_jsonl(out, {"experiment_id": "E6_network", "label": "local", "timestamp_utc": utc_now(), "document_id": it["document_id"], "target_tokens": L, "prefix_tokens": len(pre), "query_tokens": len(query), "repeat": rep,
74+ "method": "local_prefill", "status": "success", "answer": g["text"], "correct": score_synthetic(g["text"], qk)["correct"], "local_prefill_ms": tp["prefill_ms"], "local_ttft_ms": g["ttft_ms"], "local_decode_ms": g["decode_ms"]})
75+cl.close()
76+print("done ->", out)
modified src/wp30/backends/mlx_backend.py +10 −1
@@ -160,6 +160,15 @@ class MLXModel:
160 out_ids: list[int] = [] 160 out_ids: list[int] = []
161 t.start("query_prefill") 161 t.start("query_prefill")
162 toks = mx.array(query_ids, dtype=mx.int32) 162 toks = mx.array(query_ids, dtype=mx.int32)
163+ # Heterogeneous retained lengths across layers (e.g. PyramidKV budgets) cannot share one attention mask
164+ # (mlx_lm builds the mask from cache[0]); process the query tokens one at a time in that case (mask=None).
165+ kept = {getattr(c, "n_kept", None) for c in cache}
166+ heterogeneous = len(kept) > 1
167+ if heterogeneous and len(query_ids) > 1:
168+ for i in range(len(query_ids) - 1):
169+ self.model(toks[i:i + 1][None], cache=cache)
170+ mx.eval([c.keys for c in cache])
171+ toks = toks[-1:]
163 logits = self.model(toks[None], cache=cache)[:, -1, :] 172 logits = self.model(toks[None], cache=cache)[:, -1, :]
164 logprobs = logits - mx.logsumexp(logits, keepdims=True) 173 logprobs = logits - mx.logsumexp(logits, keepdims=True)
165 y = sampler(logprobs) 174 y = sampler(logprobs)
@@ -186,7 +195,7 @@ class MLXModel:
186 mx.synchronize() 195 mx.synchronize()
187 t.stop("decode") 196 t.stop("decode")
188 text = self.decode(out_ids) 197 text = self.decode(out_ids)
189− return {"text": text, "output_ids": out_ids, "output_tokens": len(out_ids), "ttft_ms": ttft_ms, 198+ return {"text": text, "output_ids": out_ids, "output_tokens": len(out_ids), "ttft_ms": ttft_ms, "sequential_query_prefill": heterogeneous,
190 "query_prefill_ms": t.spans["query_prefill"], "decode_ms": t.spans["decode"], "query_tokens": len(query_ids)} 199 "query_prefill_ms": t.spans["query_prefill"], "decode_ms": t.spans["decode"], "query_tokens": len(query_ids)}
191 200
192 def generate_text(self, token_ids: list[int], max_tokens: int = 64, temp: float = 0.0) -> dict: 201 def generate_text(self, token_ids: list[int], max_tokens: int = 64, temp: float = 0.0) -> dict:
added src/wp30/chains.py +159 −0
@@ -0,0 +1,159 @@
1+"""E4: repeated handoffs (relay chains) with an evaluator-only fact ledger.
2+
3+Chain: agent_1 reads the document and writes m_1 (A4 prose or A5 structured, question-blind). agent_i (i>1) reads only
4+m_{i-1} (closed) or m_{i-1} plus metered source lookups (repairable) and writes m_i under the same budget. After hops
5+h in HOPS the receiver answers all questions from m_h. Retention/distortion is scored from the ledger (never shown to agents).
6+"""
7+from __future__ import annotations
8+import json, re, time, yaml
9+from pathlib import Path
10+import mlx.core as mx
11+from .util import ROOT, read_json, sha256_json, sha256_text, utc_now, read_jsonl
12+from .records import new_record, write_record, trial_key
13+from .data.prepare import load_doc, load_key
14+from .scoring import score_synthetic, normalize_answer, extract_numbers
15+from .prompts import SYSTEM_RECEIVER, receiver_user_text, SUMMARY_SYSTEM, EVIDENCE_SYSTEM
16+from .backends.mlx_backend import MLXModel, reset_peak
17+from .methods import text as TM
18+
19+HARDWARE_ID = "laptop-m5max-48g"
20+
21+RELAY_SUMMARY_USER = ("You received the following notes from a colleague about a document you cannot see. Rewrite them as a faithful summary of at most "
22+ "{budget} tokens for the next colleague, preserving every concrete fact (names, identifiers, numbers, units, dates, statuses, negations) "
23+ "exactly as written. Do not add facts that are not in the notes.{repair}\n\nNotes:\n{msg}\n\nSummary:")
24+RELAY_EVIDENCE_USER = ("You received the following structured evidence (JSON) from a colleague about a document you cannot see. Re-emit it as a JSON object with the same "
25+ "schema (keys facts, optional notes, optional unknowns), under {budget} tokens, keeping every fact and value exactly; drop nothing unless the "
26+ "budget forces you to, and never invent values.{repair}\n\nEvidence:\n{msg}\n\nJSON:")
27+
28+def _ledger_metrics(message: str, facts: list[dict], doc_text: str) -> dict:
29+ """Retention: a fact counts as retained if its value (normalized) appears in the message. Unsupported additions:
30+ numbers with >=4 significant digits, identifiers (XX-00X-0000) and 4-digit-year dates in the message that do not
31+ occur in the document."""
32+ msg_n = normalize_answer(message); msg_nc = message.replace(",", "")
33+ retained = 0; by_kind = {}
34+ for f in facts:
35+ v = f["value"]
36+ ok = normalize_answer(v) in msg_n or v.replace(",", "") in msg_nc
37+ by_kind.setdefault(f["kind"], [0, 0]); by_kind[f["kind"]][1] += 1
38+ if ok:
39+ retained += 1; by_kind[f["kind"]][0] += 1
40+ doc_nums = {str(int(x)) for x in extract_numbers(doc_text) if abs(x) >= 1000}
41+ msg_nums = {str(int(x)) for x in extract_numbers(message) if abs(x) >= 1000}
42+ unsupported_nums = msg_nums - doc_nums
43+ doc_ids = set(re.findall(r"\b[A-Z]{2}-\d{2}[A-Z]-\d{4}\b", doc_text)); msg_ids = set(re.findall(r"\b[A-Z]{2}-\d{2}[A-Z]-\d{4}\b", message))
44+ unsupported_ids = msg_ids - doc_ids
45+ # distorted amounts: a project code co-occurring (same sentence) with an unsupported number
46+ distorted = 0
47+ for sent in re.split(r"(?<=[.!?])\s+|\n+", message):
48+ if re.search(r"\b[A-Z][a-z]+-\d{1,2}\b", sent) and any(n in sent.replace(",", "") for n in unsupported_nums):
49+ distorted += 1
50+ return {"facts_total": len(facts), "facts_retained": retained, "retention_rate": retained / max(1, len(facts)),
51+ "retention_by_kind": {k: v[0] / v[1] for k, v in by_kind.items()},
52+ "unsupported_numbers": len(unsupported_nums), "unsupported_identifiers": len(unsupported_ids), "distorted_sentences": distorted,
53+ "message_bytes": len(message.encode("utf-8"))}
54+
55+def _repair_lookup(prev_msg: str, doc_text: str, max_bytes: int, max_lookups: int) -> tuple[str, dict]:
56+ """Repairable chain: the agent re-reads source sentences for entity codes mentioned in the notes (metered)."""
57+ codes = []
58+ for c in re.findall(r"\b[A-Z][a-z]+-\d{1,2}\b", prev_msg):
59+ if c not in codes: codes.append(c)
60+ sents = re.split(r"(?<=[.!?])\s+", doc_text)
61+ fetched, nb, n = [], 0, 0
62+ for c in codes:
63+ if n >= max_lookups: break
64+ hits = [s for s in sents if c in s]
65+ if not hits: continue
66+ chunk = " ".join(hits)
67+ b = len(chunk.encode("utf-8"))
68+ if nb + b > max_bytes: continue
69+ fetched.append(f"[source passages for {c}] {chunk}"); nb += b; n += 1
70+ return "\n".join(fetched), {"lookups": n, "fetched_bytes": nb, "codes_mentioned": len(codes)}
71+
72+def run_chains(config_path: str, resume: bool = False) -> int:
73+ cfg = yaml.safe_load(open(config_path))
74+ mcfg = yaml.safe_load(open(ROOT / "configs/models.yaml"))["models"]
75+ models = {}
76+ def get(a):
77+ if a not in models: models[a] = MLXModel(mcfg[a]["repo"])
78+ return models[a]
79+ man = read_json(ROOT / "data/manifests/splits.json")
80+ items = [it for it in man["items"] if it["family"] == "synthetic" and it["split"] in cfg["splits"] and it.get("target_tokens") in cfg["lengths"]]
81+ if cfg.get("limit_docs"): items = items[: cfg["limit_docs"]]
82+ out_path = ROOT / "runs/raw" / f"{cfg['experiment_id']}.jsonl"
83+ log_path = ROOT / "runs/logs" / f"{cfg['experiment_id']}.log"
84+ done = {r["trial_id"] for r in read_jsonl(out_path) if r["status"] == "success"} if (resume and out_path.exists()) else set()
85+ protocol_hash = cfg.get("protocol_hash", "unfrozen"); config_hash = sha256_json(cfg)
86+ def log(m):
87+ line = f"{utc_now()} {m}"; print(line, flush=True)
88+ with open(log_path, "a") as f: f.write(line + "\n")
89+ log(f"chains start {cfg['experiment_id']} items={len(items)} done={len(done)}")
90+ HOPS = cfg["hops"]; budget = cfg["budget_tokens"]
91+ for it in items:
92+ doc = load_doc(it["document_id"], "synthetic"); key = load_key(it["document_id"], "synthetic")
93+ qkeys = {q["question_id"]: q for q in key["questions"]}
94+ for chain in cfg["chains"]:
95+ rep, mode, agents = chain["representation"], chain["mode"], chain["agents"] # agents: list of aliases cycled
96+ cid = f"{rep}|{mode}|{'-'.join(agents)}"
97+ # skip if all hop trials done
98+ keys_needed = [trial_key(cfg["experiment_id"], it["document_id"], q["question_id"], rep, {"mode": mode, "agents": agents, "budget": budget}, agents[0], agents[-1], "blind", protocol_hash, 0, "chain", h) for h in HOPS for q in doc["questions"]]
99+ if all(k in done for k in keys_needed):
100+ continue
101+ msg = None; hop_msgs = {}; hop_meta = {}
102+ t_chain = 0.0; fetched_total = 0; lookups_total = 0
103+ for h in range(1, max(HOPS) + 1):
104+ agent = get(agents[(h - 1) % len(agents)])
105+ t0 = time.perf_counter()
106+ repair_txt, rinfo = "", {"lookups": 0, "fetched_bytes": 0}
107+ if h == 1:
108+ m = TM.a4_summary(doc["text"], budget, agent, None) if rep == "A4" else TM.a5_structured(doc["text"], budget, agent, None)
109+ text = m.text; usage = m.sender_usage
110+ else:
111+ if mode == "repairable":
112+ repair_txt, rinfo = _repair_lookup(msg, doc["text"], cfg.get("max_fetch_bytes", 1500), cfg.get("max_lookups", 4))
113+ rep_clause = ("\n\nYou may also use these passages re-read from the original source:\n" + repair_txt) if repair_txt else ""
114+ if rep == "A4":
115+ user = RELAY_SUMMARY_USER.format(budget=budget, msg=msg, repair=rep_clause)
116+ text, usage, _ = TM._sender_generate(agent, SUMMARY_SYSTEM, user, budget)
117+ else:
118+ user = RELAY_EVIDENCE_USER.format(budget=budget, msg=msg, repair=rep_clause)
119+ raw, usage, _ = TM._sender_generate(agent, EVIDENCE_SYSTEM, user, budget)
120+ obj = TM._parse_json_loose(raw)
121+ text = json.dumps(obj, ensure_ascii=False) if isinstance(obj, dict) else raw
122+ hop_ms = (time.perf_counter() - t0) * 1000; t_chain += hop_ms
123+ fetched_total += rinfo["fetched_bytes"]; lookups_total += rinfo["lookups"]
124+ msg = text; hop_msgs[h] = text
125+ hop_meta[h] = {"hop_ms": hop_ms, "agent": agent.info.repo_id, "usage": usage, "repair": rinfo, "ledger": _ledger_metrics(text, key["facts"], doc["text"])}
126+ if h in HOPS:
127+ receiver = get(agents[(h - 1) % len(agents)]) # the last agent in the chain answers
128+ for q in doc["questions"]:
129+ tk = trial_key(cfg["experiment_id"], it["document_id"], q["question_id"], rep, {"mode": mode, "agents": agents, "budget": budget}, agents[0], agents[-1], "blind", protocol_hash, 0, "chain", h)
130+ if tk in done: continue
131+ rec = new_record(experiment_id=cfg["experiment_id"], trial_id=tk, run_group_id=f"{cfg['experiment_id']}:{cid}:h{h}", protocol_hash=protocol_hash, config_hash=config_hash,
132+ task_family="synthetic", document_id=it["document_id"], document_hash=it["document_hash"], question_id=q["question_id"], split=it["split"], cluster_id=it["cluster_id"],
133+ sender_model_revision="|".join(f"{get(a).info.repo_id}@{get(a).info.revision}" for a in agents), receiver_model_revision=f"{receiver.info.repo_id}@{receiver.info.revision}",
134+ tokenizer_hashes={"receiver": receiver.info.tokenizer_hash}, backend_versions={"mlx": mx.__version__, "mlx_lm": receiver.info.mlx_lm_version}, hardware_id=HARDWARE_ID,
135+ device_state={}, weight_precision="bf16", method_id=rep, implementation_fidelity="custom", compression_config={"mode": mode, "agents": agents, "budget_tokens": budget},
136+ question_access="blind", topology=f"chain_{mode}", handoff_index=h, reuse_count=len(doc["questions"]), cache_hit=False, cache_compatibility_hash=None, random_seed=0,
137+ source_tokens_sender=None, source_tokens_receiver=len(receiver.encode(doc["text"])), message_utf8_bytes=len(text.encode("utf-8")), serialized_payload_bytes=len(text.encode("utf-8")),
138+ cache_tensor_bytes=None, side_information_bytes=0, fetched_bytes=fetched_total, timing_spans={"chain_ms": t_chain, "hops": {str(k): v["hop_ms"] for k, v in hop_meta.items()}},
139+ input_usage={"lookups": lookups_total}, output_usage={}, provider_cached_usage=None, process_memory=None, backend_memory=None, peak_prefill_memory=None, peak_decode_memory=None,
140+ mapper_memory=None, cost_with_source=None, evidence_artifact={"ledger": hop_meta[h]["ledger"], "ledger_by_hop": {str(k): v["ledger"] for k, v in hop_meta.items()}},
141+ reference_key_id=f"keys/synthetic/{it['document_id']}.json", status="implementation_error", error_code=None, log_artifact=str(log_path.relative_to(ROOT)), payload_hash=sha256_text(text), output_hash=None)
142+ try:
143+ ids = receiver.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text("Notes received after %d handoff(s)" % h, text, q["question"], "synthetic")}])
144+ t1 = time.perf_counter(); g = receiver.generate_text(ids, max_tokens=48)
145+ rec.update({"prediction_artifact": g["text"], "output_hash": sha256_text(g["text"]), "query_tokens": len(ids), "message_tokens": len(receiver.encode(text)),
146+ "ttft_ms": g["ttft_ms"], "total_wall_ms": (time.perf_counter() - t1) * 1000 + t_chain, "output_usage": {"output_tokens": g["output_tokens"]},
147+ "task_scores": score_synthetic(g["text"], qkeys[q["question_id"]]), "status": "success" if g["text"].strip() else "invalid_output"})
148+ rec["timing_spans"].update({"receiver_prefill_ms": g["prefill_ms"], "ttft_ms": g["ttft_ms"], "decode_ms": g["decode_ms"]})
149+ except Exception as e:
150+ rec["error_code"] = f"{type(e).__name__}: {str(e)[:200]}"
151+ write_record(rec, out_path)
152+ L = hop_meta[h]["ledger"]
153+ log(f"{cid:>28s} {it['document_id']} hop={h} retention={L['retention_rate']:.2f} unsupported={L['unsupported_numbers']}+{L['unsupported_identifiers']} bytes={L['message_bytes']} fetched={fetched_total}")
154+ # store hop messages for audit
155+ Path(ROOT / "runs/raw/chains").mkdir(parents=True, exist_ok=True)
156+ (ROOT / "runs/raw/chains" / f"{cfg['experiment_id']}_{it['document_id']}_{cid.replace('|','_').replace('-','_')}.json").write_text(json.dumps({"hops": hop_msgs, "meta": hop_meta}, ensure_ascii=False, indent=1, default=str))
157+ mx.clear_cache()
158+ log("chains end")
159+ return 0
added src/wp30/cli_extra.py +14 −0
@@ -0,0 +1,14 @@
1+import sys
2+def main():
3+ cmd = sys.argv[1]; cfg = sys.argv[2]; resume = "--resume" in sys.argv
4+ if cmd == "chains":
5+ from .chains import run_chains; return run_chains(cfg, resume)
6+ if cmd == "fanin":
7+ from .fanin import run_fanin; return run_fanin(cfg, resume)
8+ if cmd == "router":
9+ from .router import main as f; return f(cfg)
10+ if cmd == "hosted":
11+ from .hosted_run import run as f; return f(cfg, resume)
12+ return 2
13+if __name__ == "__main__":
14+ sys.exit(main())
modified src/wp30/data/exec_tasks.py +6 −9
@@ -31,9 +31,8 @@ def spec_variants(rng: random.Random, rows):
31 spec = (f"Bug report BR-{rng.randint(1000,9999)}: the function net_revenue(rows) must compute the total net revenue of the " 31 spec = (f"Bug report BR-{rng.randint(1000,9999)}: the function net_revenue(rows) must compute the total net revenue of the "
32 f"rows list (each row is a dict with keys {COLS}). Rules agreed with the client on {rng.randint(1,28)} {rng.choice(['March','June','September'])} 2031: " 32 f"rows list (each row is a dict with keys {COLS}). Rules agreed with the client on {rng.randint(1,28)} {rng.choice(['March','June','September'])} 2031: "
33 f"(1) exclude returned rows; (2) exclude rows whose region is '{excl_region}'; (3) exclude rows with fewer than {min_units} units; " 33 f"(1) exclude returned rows; (2) exclude rows whose region is '{excl_region}'; (3) exclude rows with fewer than {min_units} units; "
34− f"(4) rows shipped in more than {thr_days} days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; " 34+ f"(4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of {tax} % on the sum; "
35− f"(5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of {tax} % on the sum; " 35+ f"(6) return the result rounded to {rounding} decimals (Python round). The current implementation forgets rule (2) and applies no tax.")
36− f"(7) return the result rounded to {rounding} decimals (Python round). The current implementation forgets rule (4) and rule (2).")
37 ref = textwrap.dedent(f""" 36 ref = textwrap.dedent(f"""
38 def net_revenue(rows): 37 def net_revenue(rows):
39 total = 0.0 38 total = 0.0
@@ -44,9 +43,7 @@ def spec_variants(rng: random.Random, rows):
44 continue 43 continue
45 if r['units'] < {min_units}: 44 if r['units'] < {min_units}:
46 continue 45 continue
47− d = r['discount_pct'] + (5 if r['shipped_days'] > {thr_days} else 0) 46+ total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)
48− d = min(d, 50)
49− total += r['units'] * r['unit_price'] * (1 - d / 100.0)
50 total = total * (1 + {tax} / 100.0) 47 total = total * (1 + {tax} / 100.0)
51 return round(total, {rounding}) 48 return round(total, {rounding})
52 """) 49 """)
@@ -77,9 +74,9 @@ def generate_exec_task(seed: int, target_tokens: int = 2000) -> dict:
77 doc = ("ENGINEERING TICKET\n\n" + spec + "\n\nSample rows (first 6 of the extract):\n" + sample_rows + "\n\n" + 74 doc = ("ENGINEERING TICKET\n\n" + spec + "\n\nSample rows (first 6 of the extract):\n" + sample_rows + "\n\n" +
78 "\n".join(filler[: n_fill // 2]) + "\n\nThe existing (buggy) implementation is:\n\n" + 75 "\n".join(filler[: n_fill // 2]) + "\n\nThe existing (buggy) implementation is:\n\n" +
79 "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n" 76 "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n"
80− f" if r['units'] < {params['min_units']}:\n continue\n d = r['discount_pct']\n" 77+ f" if r['units'] < {params['min_units']}:\n continue\n"
81− " total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n" 78+ " total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n"
82− f" total = total * (1 + {params['tax']} / 100.0)\n return round(total, {params['rounding']})\n\n" + "\n".join(filler[n_fill // 2:])) 79+ f" return round(total, {params['rounding']})\n\n" + "\n".join(filler[n_fill // 2:]))
83 task = ("Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. " 80 task = ("Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. "
84 "Output only Python code defining net_revenue; no explanations, no markdown fences.") 81 "Output only Python code defining net_revenue; no explanations, no markdown fences.")
85 return {"document_id": f"exec_{seed:05d}_{target_tokens}", "seed": seed, "target_tokens": target_tokens, "text": doc, "char_len": len(doc), 82 return {"document_id": f"exec_{seed:05d}_{target_tokens}", "seed": seed, "target_tokens": target_tokens, "text": doc, "char_len": len(doc),
added src/wp30/fanin.py +180 −0
@@ -0,0 +1,180 @@
1+"""E5: fan-in / composition with conflicting evidence.
2+
3+The synthetic document is partitioned into K source agents (project blocks round-robin). Overlap: one project block is
4+duplicated into a second source with an *earlier* (superseded) revision sentence, so the ledger's precedence rule still
5+yields the original answers but a naive merge sees a conflict. One source additionally receives an irrelevant block from
6+another document. The aggregator answers multi-hop/aggregation/revision questions that need several sources.
7+
8+Aggregation methods: concat_A4 (concatenated prose summaries), merge_A5 (merged structured records), shared_memory_A6
9+(reference index per source + metered retrieval by the aggregator), full_context (reference: all source texts concatenated),
10+and cache_concat_negative_control (KV caches of independently prefilled sources concatenated along the position axis —
11+positions overlap and no cross-source context exists; reported ONLY as a labelled negative control).
12+"""
13+from __future__ import annotations
14+import json, re, time, yaml, random
15+from pathlib import Path
16+import numpy as np
17+import mlx.core as mx
18+from .util import ROOT, read_json, sha256_json, sha256_text, utc_now, read_jsonl
19+from .records import new_record, write_record, trial_key
20+from .data.prepare import load_doc, load_key
21+from .data.synthetic import fmt_amt
22+from .scoring import score_synthetic
23+from .prompts import SYSTEM_RECEIVER, receiver_user_text
24+from .backends.mlx_backend import MLXModel
25+from .cache.sparse import SparseKVCache, cache_tensor_bytes, dense_kv
26+from .methods import text as TM
27+
28+HARDWARE_ID = "laptop-m5max-48g"
29+
30+def partition_document(doc_text: str, key: dict, k: int, rng: random.Random, other_doc_text: str) -> list[dict]:
31+ blocks = doc_text.split("\n\n")
32+ header = blocks[0]; review = blocks[1]
33+ proj = [b for b in blocks[2:] if b.startswith("Project ")]
34+ fill = [b for b in blocks[2:] if not b.startswith("Project ")]
35+ sources = [{"source_id": f"S{i+1}", "blocks": [header]} for i in range(k)]
36+ for i, b in enumerate(proj):
37+ sources[i % k]["blocks"].append(b)
38+ for i, b in enumerate(fill):
39+ sources[i % k]["blocks"].append(b)
40+ # overlap with superseded earlier revision: pick a project with >=1 revision
41+ rev_facts = [f for f in key["facts"] if f["attribute"].startswith("revision_1")]
42+ conflict = None
43+ if rev_facts and k >= 2:
44+ f = rev_facts[0]; code = f["entity"]
45+ blk = next((b for b in proj if b.startswith(f"Project {code} ")), None)
46+ if blk:
47+ # earlier date than the first revision: use the proposed date + 1 day (always earlier than revisions)
48+ m = re.search(rf"Project {re.escape(code)} was proposed by .*? on (\d+ \w+ \d{{4}})", blk)
49+ amt = int(re.sub(r"[^\d]", "", f["value"].split()[0])); bogus = fmt_amt(int(amt * 0.93) // 5 * 5)
50+ cur = f["value"].split()[-1]
51+ earlier = f"On {m.group(1)} (preliminary) the budget of {code} was revised to {bogus} {cur}; this preliminary entry is superseded by any later-dated entry."
52+ tgt = (proj.index(blk) + 1) % k
53+ sources[tgt]["blocks"].append(blk.split(". ")[0] + ". " + earlier)
54+ conflict = {"code": code, "bogus_amount": bogus, "source": sources[tgt]["source_id"]}
55+ # irrelevant block from another document
56+ other_proj = [b for b in other_doc_text.split("\n\n") if b.startswith("Project ")]
57+ if other_proj:
58+ sources[-1]["blocks"].append(other_proj[0])
59+ for s in sources:
60+ s["text"] = "\n\n".join(s["blocks"])
61+ return sources, conflict
62+
63+def cache_concat_negative_control(receiver: MLXModel, source_texts: list[str]):
64+ """Prefill each source independently (positions 0..T_i), then concatenate K/V along positions. Overlapping positions
65+ and no cross-source attention during prefill make this an INVALID composition; measured as a negative control."""
66+ caches = []
67+ for t in source_texts:
68+ ids = receiver.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": "Document:\n" + t + "\n\n"}])
69+ c, _ = receiver.prefill(ids); caches.append(c)
70+ out = []
71+ for layer in range(len(caches[0])):
72+ ks = [dense_kv(c[layer])[0] for c in caches]; vs = [dense_kv(c[layer])[1] for c in caches]
73+ k = mx.concatenate(ks, axis=2); v = mx.concatenate(vs, axis=2)
74+ out.append(SparseKVCache.from_dense(k, v, logical_offset=max(c[layer].offset for c in caches)))
75+ return out
76+
77+def run_fanin(config_path: str, resume: bool = False) -> int:
78+ cfg = yaml.safe_load(open(config_path))
79+ mcfg = yaml.safe_load(open(ROOT / "configs/models.yaml"))["models"]
80+ models = {}
81+ def get(a):
82+ if a not in models: models[a] = MLXModel(mcfg[a]["repo"])
83+ return models[a]
84+ man = read_json(ROOT / "data/manifests/splits.json")
85+ items = [it for it in man["items"] if it["family"] == "synthetic" and it["split"] in cfg["splits"] and it.get("target_tokens") in cfg["lengths"]]
86+ if cfg.get("limit_docs"): items = items[: cfg["limit_docs"]]
87+ out_path = ROOT / "runs/raw" / f"{cfg['experiment_id']}.jsonl"; log_path = ROOT / "runs/logs" / f"{cfg['experiment_id']}.log"
88+ done = {r["trial_id"] for r in read_jsonl(out_path) if r["status"] == "success"} if (resume and out_path.exists()) else set()
89+ protocol_hash = cfg.get("protocol_hash", "unfrozen"); config_hash = sha256_json(cfg)
90+ def log(m):
91+ line = f"{utc_now()} {m}"; print(line, flush=True)
92+ with open(log_path, "a") as f: f.write(line + "\n")
93+ QTYPES = set(cfg.get("question_types", ["multi_hop", "aggregation", "revision", "temporal", "unanswerable"]))
94+ agg = get(cfg["aggregator"]); sender = get(cfg["sender"])
95+ for idx, it in enumerate(items):
96+ doc = load_doc(it["document_id"], "synthetic"); key = load_key(it["document_id"], "synthetic")
97+ other = load_doc(items[(idx + 1) % len(items)]["document_id"], "synthetic")
98+ qkeys = {q["question_id"]: q for q in key["questions"]}
99+ qs = [q for q in doc["questions"] if q["qtype"] in QTYPES]
100+ for k in cfg["fan_in"]:
101+ rng = random.Random(it["seed"] * 7 + k)
102+ sources, conflict = partition_document(doc["text"], key, k, rng, other["text"])
103+ for method in cfg["methods"]:
104+ cid = f"{method}|k={k}"
105+ tks = {q["question_id"]: trial_key(cfg["experiment_id"], it["document_id"], q["question_id"], method, {"k": k, "budget": cfg["budget_tokens"]}, cfg["sender"], cfg["aggregator"], "blind", protocol_hash, 0, "fanin") for q in qs}
106+ if all(t in done for t in tks.values()): continue
107+ t0 = time.perf_counter(); info = {"k": k, "conflict": conflict}
108+ try:
109+ cache = None; material = None; per_q = False
110+ if method == "full_context":
111+ material = "\n\n".join(f"[Source {s['source_id']}]\n{s['text']}" for s in sources); label = "Documents from all sources"
112+ elif method == "concat_A4":
113+ msgs = [TM.a4_summary(s["text"], cfg["budget_tokens"], sender, None) for s in sources]
114+ material = "\n\n".join(f"[Source {s['source_id']} summary]\n{m.text}" for s, m in zip(sources, msgs)); label = "Summaries from several sources"
115+ info["sender_tokens"] = sum(m.sender_usage.get("sender_output_tokens", 0) for m in msgs)
116+ elif method == "merge_A5":
117+ msgs = [TM.a5_structured(s["text"], cfg["budget_tokens"], sender, None) for s in sources]
118+ facts = []
119+ for s, m in zip(sources, msgs):
120+ for f in json.loads(m.text).get("facts", []):
121+ facts.append({"source": s["source_id"], **{kk: vv for kk, vv in f.items() if kk != "span"}})
122+ material = json.dumps({"facts": facts, "rule": "later-dated entries supersede earlier ones"}, ensure_ascii=False); label = "Merged structured evidence from several sources (JSON)"
123+ info["n_facts"] = len(facts)
124+ elif method == "shared_memory_A6":
125+ msgs = [TM.a6_references(s["text"], cfg["budget_tokens"], sender, None) for s in sources]
126+ per_q = True; label = "Reference index from several sources plus retrieved passages"
127+ elif method == "cache_concat_negative_control":
128+ cache = cache_concat_negative_control(agg, [s["text"] for s in sources]); label = None
129+ info["cache_bytes"] = cache_tensor_bytes(cache)
130+ else:
131+ raise NotImplementedError(method)
132+ build_ms = (time.perf_counter() - t0) * 1000
133+ except Exception as e:
134+ log(f"BUILD FAIL {cid} {it['document_id']}: {type(e).__name__}: {str(e)[:200]}"); continue
135+ for q in qs:
136+ tk = tks[q["question_id"]]
137+ if tk in done: continue
138+ rec = new_record(experiment_id=cfg["experiment_id"], trial_id=tk, run_group_id=f"{cfg['experiment_id']}:{cid}", protocol_hash=protocol_hash, config_hash=config_hash,
139+ task_family="synthetic", document_id=it["document_id"], document_hash=it["document_hash"], question_id=q["question_id"], split=it["split"], cluster_id=it["cluster_id"],
140+ sender_model_revision=f"{sender.info.repo_id}@{sender.info.revision}", receiver_model_revision=f"{agg.info.repo_id}@{agg.info.revision}", tokenizer_hashes={"receiver": agg.info.tokenizer_hash},
141+ backend_versions={"mlx": mx.__version__, "mlx_lm": agg.info.mlx_lm_version}, hardware_id=HARDWARE_ID, device_state={}, weight_precision="bf16", method_id=method,
142+ implementation_fidelity="custom" if method != "cache_concat_negative_control" else "n/a", compression_config={"k": k, "budget_tokens": cfg["budget_tokens"], **{kk: vv for kk, vv in info.items() if kk != "conflict"}},
143+ question_access="blind", topology=f"fanin_k{k}", handoff_index=1, reuse_count=len(qs), cache_hit=False, cache_compatibility_hash=None, random_seed=0,
144+ source_tokens_sender=None, source_tokens_receiver=len(agg.encode(doc["text"])), fetched_bytes=0, side_information_bytes=0, timing_spans={"build_ms": build_ms},
145+ input_usage={}, output_usage={}, provider_cached_usage=None, process_memory=None, backend_memory=None, peak_prefill_memory=None, peak_decode_memory=None, mapper_memory=None,
146+ cost_with_source=None, evidence_artifact={"conflict": conflict, "question_type": q["qtype"]}, reference_key_id=f"keys/synthetic/{it['document_id']}.json", status="implementation_error",
147+ error_code=None, log_artifact=str(log_path.relative_to(ROOT)), payload_hash=None, output_hash=None, message_utf8_bytes=None, serialized_payload_bytes=None, cache_tensor_bytes=info.get("cache_bytes"), query_tokens=None, message_tokens=None)
148+ try:
149+ t1 = time.perf_counter()
150+ if cache is not None:
151+ from .cache.sparse import copy_cache
152+ # query tokens = the fresh tail of a full prompt built on the last source (identical tail for every source)
153+ full_ids = agg.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text("Document", sources[-1]["text"], q["question"], "synthetic")}])
154+ pre_ids = agg.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": "Document:\n" + sources[-1]["text"] + "\n\n"}])
155+ n = 0
156+ while n < min(len(pre_ids), len(full_ids)) and pre_ids[n] == full_ids[n]: n += 1
157+ q_ids = full_ids[n:]
158+ g = agg.generate_from_cache(copy_cache(cache), q_ids, max_tokens=48)
159+ rec.update({"serialized_payload_bytes": info["cache_bytes"], "message_utf8_bytes": sum(len(s["text"].encode()) for s in sources), "query_tokens": len(q_ids)})
160+ else:
161+ mat = material
162+ if per_q:
163+ pieces = []; fb = 0; lk = 0
164+ for m in msgs:
165+ mt, ri = TM.a6_retrieve(m, q["question"], cfg.get("max_fetch_bytes", 1500), cfg.get("max_lookups", 3)); pieces.append(mt); fb += ri["fetched_bytes"]; lk += ri["lookups"]
166+ mat = "\n\n".join(pieces); rec["fetched_bytes"] = fb; rec["input_usage"] = {"lookups": lk}
167+ ids = agg.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text(label, mat, q["question"], "synthetic")}])
168+ g = agg.generate_text(ids, max_tokens=48)
169+ rec.update({"serialized_payload_bytes": len(mat.encode("utf-8")), "message_utf8_bytes": len(mat.encode("utf-8")), "query_tokens": len(ids), "message_tokens": len(agg.encode(mat)), "payload_hash": sha256_text(mat)})
170+ rec.update({"prediction_artifact": g["text"], "output_hash": sha256_text(g["text"]), "ttft_ms": g["ttft_ms"], "total_wall_ms": (time.perf_counter() - t1) * 1000 + build_ms,
171+ "output_usage": {"output_tokens": g["output_tokens"]}, "task_scores": score_synthetic(g["text"], qkeys[q["question_id"]]), "status": "success" if g["text"].strip() else "invalid_output"})
172+ rec["timing_spans"].update({"ttft_ms": g["ttft_ms"], "decode_ms": g["decode_ms"], "receiver_prefill_ms": g.get("prefill_ms", 0.0)})
173+ except Exception as e:
174+ rec["error_code"] = f"{type(e).__name__}: {str(e)[:200]}"
175+ write_record(rec, out_path)
176+ sc = rec["task_scores"] or {}
177+ log(f"{cid:>36s} {it['document_id']} {q['question_id']:>18s} {rec['status']:>8s} corr={sc.get('correct')} bytes={rec['serialized_payload_bytes']} ans={json.dumps((rec['prediction_artifact'] or '')[:50])}")
178+ mx.clear_cache()
179+ log("fanin end")
180+ return 0
modified src/wp30/runner.py +14 −6
@@ -90,18 +90,24 @@ class DocContext:
90 self._dense = None; self.build_ms = None; self.prefix_ids = None; self.full_ids_fn = None 90 self._dense = None; self.build_ms = None; self.prefix_ids = None; self.full_ids_fn = None
91 self.sys = SYSTEM_RECEIVER 91 self.sys = SYSTEM_RECEIVER
92 def prefix_tokens(self): 92 def prefix_tokens(self):
93+ """Receiver prefix = chat tokens up to (and including) the document and the blank line before 'Question:'.
94+ Computed as the longest common token prefix between the prefix text and a full prompt with a canonical question."""
93 if self.prefix_ids is None: 95 if self.prefix_ids is None:
94 pre_txt = self.receiver.tokenizer.apply_chat_template([{"role": "system", "content": self.sys}, {"role": "user", "content": "Document:\n" + self.doc_text + "\n\n"}], add_generation_prompt=False, tokenize=False) 96 pre_txt = self.receiver.tokenizer.apply_chat_template([{"role": "system", "content": self.sys}, {"role": "user", "content": "Document:\n" + self.doc_text + "\n\n"}], add_generation_prompt=False, tokenize=False)
95− self.prefix_ids = self.receiver.encode(pre_txt) 97+ pre = self.receiver.encode(pre_txt)
98+ full = self.receiver.chat_prompt_tokens([{"role": "system", "content": self.sys}, {"role": "user", "content": receiver_user_text("Document", self.doc_text, "What is stated?", self.family)}])
99+ n = 0
100+ while n < min(len(pre), len(full)) and pre[n] == full[n]:
101+ n += 1
102+ self.prefix_ids = full[:n]
96 return self.prefix_ids 103 return self.prefix_ids
97 def split_ids(self, question: str): 104 def split_ids(self, question: str):
98 """Full chat tokens for (doc, question); returns (prefix_ids, query_ids) with prefix a true token prefix.""" 105 """Full chat tokens for (doc, question); returns (prefix_ids, query_ids) with prefix a true token prefix."""
99 full = self.receiver.chat_prompt_tokens([{"role": "system", "content": self.sys}, {"role": "user", "content": receiver_user_text("Document", self.doc_text, question, self.family)}]) 106 full = self.receiver.chat_prompt_tokens([{"role": "system", "content": self.sys}, {"role": "user", "content": receiver_user_text("Document", self.doc_text, question, self.family)}])
100 pre = self.prefix_tokens() 107 pre = self.prefix_tokens()
101− n = 0 108+ if full[:len(pre)] != pre:
102− while n < min(len(pre), len(full)) and pre[n] == full[n]: 109+ raise RuntimeError("prefix_boundary: question tokenization altered the document prefix")
103− n += 1 110+ return pre, full[len(pre):]
104− return full[:n], full[n:]
105 def dense_cache(self, prefix_ids): 111 def dense_cache(self, prefix_ids):
106 if self._dense is None or self._dense_len != len(prefix_ids): 112 if self._dense is None or self._dense_len != len(prefix_ids):
107 reset_peak() 113 reset_peak()
@@ -130,7 +136,8 @@ def run(config_path: str, resume: bool = False, limit: int | None = None, dry_ru
130 done.add(r["trial_id"]) 136 done.add(r["trial_id"])
131 items = _select_items(cfg) 137 items = _select_items(cfg)
132 get_model, mcfg = _load_models(cfg) 138 get_model, mcfg = _load_models(cfg)
133− gen_max = cfg.get("generation", {}).get("max_tokens", 48) 139+ gen_cfg = cfg.get("generation", {})
140+ gen_max = gen_cfg.get("max_tokens", 48)
134 n_done = n_run = n_fail = 0 141 n_done = n_run = n_fail = 0
135 log_path = ROOT / "runs/logs" / f"{cfg['experiment_id']}.log" 142 log_path = ROOT / "runs/logs" / f"{cfg['experiment_id']}.log"
136 log_path.parent.mkdir(parents=True, exist_ok=True) 143 log_path.parent.mkdir(parents=True, exist_ok=True)
@@ -145,6 +152,7 @@ def run(config_path: str, resume: bool = False, limit: int | None = None, dry_ru
145 for it in items: 152 for it in items:
146 fam = it["family"]; doc = load_doc(it["document_id"], fam); key = load_key(it["document_id"], fam) 153 fam = it["family"]; doc = load_doc(it["document_id"], fam); key = load_key(it["document_id"], fam)
147 qs = doc["questions"][: cfg.get("max_questions_per_doc") or None] 154 qs = doc["questions"][: cfg.get("max_questions_per_doc") or None]
155+ gen_max = gen_cfg.get("per_family", {}).get(fam, gen_cfg.get("max_tokens", 48))
148 qkeys = {q["question_id"]: q for q in key["questions"]} 156 qkeys = {q["question_id"]: q for q in key["questions"]}
149 for pair in cfg["model_pairs"]: 157 for pair in cfg["model_pairs"]:
150 receiver = get_model(pair["receiver"]); sender = get_model(pair["sender"]) 158 receiver = get_model(pair["receiver"]); sender = get_model(pair["sender"])
151 159
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere