SPB Git
spboucher.ai

spb/wp30_uqo

Public

WP30: runner, chains, fan-in, transport, smoke runs (Qwen, Llama), pilot plan

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Oct 30, 2026) parent fea8e12 Browse files

201 changed files +3,785 −1,269

201 changed files
Unified Split
modified DECISIONS.md +3 −0
@@ -13,3 +13,6 @@ All times America/Toronto unless noted.
13 13 - **D-007 Qwen2.5 4-bit KV finding (smoke, not a result).** Both the native mlx_lm path (`generate(..., kv_bits=4)`) and the WP30 adapter produce garbage at 4-bit/group-64 for Qwen2.5-1.5B while Llama-3.2-3B survives even 2-bit/group-32 on the smoke memo. The adapter is therefore not the cause; this is the per-token affine quantization of outlier-heavy Qwen keys (the KIVI motivation). Registered as hypothesis H2; a KIVI-inspired per-channel key quantization is scheduled as a *transport-only* variant (B2b) and labelled "inspired".
14 14 - **D-008 Task families.** (i) Synthetic private-fact ledgers (generator `wp30.data.synthetic`, 15 question types per document incl. unanswerable; evaluator-only keys); (ii) public LongBench v1 subsets qasper and 2wikimqa (all 200 items each; hotpotqa reserved as a 16K-extension subset), raw jsonl from THUDM/LongBench `data.zip` because the HF loading script is no longer supported by `datasets` 5.x; (iii) executable tasks (`wp30.data.exec_tasks`, hidden tests in a `python -I` subprocess). Family (ii) is likely in pretraining corpora; family (i) is not.
15 15 - **D-009 Remote node.** M3U96b (Mac Studio M3 Ultra, 96 GB) is the second node for genuine two-node transfer: `~/wp30-worker/.venv` with the same mlx/mlx-lm pins; both 3B checkpoints downloaded there. iperf3 available on both ends for link measurement.
16 +- **D-010 Exec family difficulty (dev-only decision, 2026-10-30).** With the 7-rule specification the receiver (Qwen2.5-3B) failed even with the full ticket (A1 pass rate 0 on the smoke item: it duplicated the loop instead of composing the discount rule), so the family could not discriminate between communication methods. The specification was reduced to 6 rules with two missing in the buggy code (region exclusion, tax). Hidden tests unchanged in form (5 subsets, exact float match). Datasets regenerated before any test-split evaluation.
17 +- **D-011 Network path for E2/E6 (2026-10-30).** The laptop cannot open TCP connections to the nodes' LAN addresses (192.168.2.x; its SSH goes through Tailscale, whose ACL blocks non-SSH ports). Genuine two-node measurements therefore run **between two cluster nodes on the wired LAN**: sender/client on M3U96a (192.168.2.80), receiver worker on M3U96b (192.168.2.85). Measured link: iperf3 4 streams ≈ 940 Mbit/s (1 GbE, not 10 GbE as older notes suggested), RTT ≈ 1.2 ms. Both nodes concurrently host a CPU-light workload from another study (WP29 `claude --bare` executors); load average ≈ 1.8 on 32 cores at setup time; recorded as a limitation.
18 +- **D-012 Learned mapper scope (Track C).** Level 2 (same family, different size): Qwen2.5-1.5B → Qwen2.5-3B share the tokenizer (token-aligned), both have 2 KV heads × 128 dims but 28 vs 36 layers → per-receiver-layer linear maps from an aligned sender layer, trained on train-split caches only (reconstruction loss; optional distillation). Level 3 (cross-family Llama-3.2-3B → Qwen2.5-3B) requires tokenizer alignment by character spans; attempted only if level 2 completes; labelled "adapted (SwaT-inspired), not a SwaT reproduction" (SwaT has no public checkpoint or licence).
modified STATUS.md +5 −4
@@ -1,7 +1,8 @@
1 1 # STATUS.md — WP30
2 2
3 −**Phase:** 0 (environment + scope) — in progress
4 −**Completed gates:** none yet
5 −**Current task:** environment inspection, literature audit launch, model download
3 +**Phase:** 3→4 complete (harness + live inference + cache correctness suite 12/12), 5 in progress (text methods implemented, smoke-tested), 6 pending
4 +**Completed gates:** Phase 0 (environment, scope), Phase 1 (literature audit: 34 sources, 3 audit files), Phase 2 (datasets: 800 items, keys isolated, 0 cross-split duplicates), Phase 3 (E0 smoke: 114 trials, real generation), Phase 4 (tests/test_cache_correctness.py 12 passed)
5 +**Current task:** pilot config, Llama receiver smoke, chain/fan-in smoke, remote worker (M3U96b) for E2/E6
6 6 **Blockers:** none
7 −**Next command:** `python -m wp30 doctor --output evidence/environment.json` (after package scaffold)
7 +**Resume:** `source .venv/bin/activate && python -m wp30 run --config configs/pilot.yaml --resume`
8 +**Run IDs so far:** E0_smoke (runs/raw/E0_smoke.jsonl)
added configs/chains_smoke.yaml +11 −0
@@ -0,0 +1,11 @@
1 +experiment_id: E4_smoke
2 +splits: [dev]
3 +lengths: [2000]
4 +limit_docs: 1
5 +hops: [1, 2]
6 +budget_tokens: 512
7 +max_fetch_bytes: 1500
8 +max_lookups: 4
9 +chains:
10 + - {representation: A4, mode: closed, agents: [qwen3b]}
11 + - {representation: A5, mode: repairable, agents: [qwen3b, llama3b]}
added configs/fanin_smoke.yaml +9 −0
@@ -0,0 +1,9 @@
1 +experiment_id: E5_smoke
2 +splits: [dev]
3 +lengths: [2000]
4 +limit_docs: 2
5 +fan_in: [2]
6 +budget_tokens: 512
7 +sender: qwen3b
8 +aggregator: qwen3b
9 +methods: [full_context, concat_A4, merge_A5, shared_memory_A6, cache_concat_negative_control]
added configs/pilot.yaml +43 −0
@@ -0,0 +1,43 @@
1 +experiment_id: E1_pilot
2 +splits: [dev]
3 +families: [synthetic, exec, public_qasper, public_2wikimqa]
4 +lengths: [2000, 8000]
5 +limit_docs_per_family: 6
6 +max_questions_per_doc: 8
7 +model_pairs:
8 + - {sender: qwen3b, receiver: qwen3b}
9 + - {sender: llama3b, receiver: llama3b}
10 +generation: {max_tokens: 48, per_family: {exec: 320}}
11 +seeds: [0]
12 +conditions:
13 + - {method: A0}
14 + - {method: A1}
15 + - {method: A2, budget_tokens: 128}
16 + - {method: A2, budget_tokens: 512}
17 + - {method: A3, budget_tokens: 128, access: blind}
18 + - {method: A3, budget_tokens: 512, access: blind}
19 + - {method: A3, budget_tokens: 512, access: aware}
20 + - {method: A4, budget_tokens: 128, access: blind}
21 + - {method: A4, budget_tokens: 512, access: blind}
22 + - {method: A4, budget_tokens: 512, access: aware}
23 + - {method: A5, budget_tokens: 128, access: blind}
24 + - {method: A5, budget_tokens: 512, access: blind}
25 + - {method: A5, budget_tokens: 512, access: aware}
26 + - {method: A6, budget_tokens: 512, access: blind, max_fetch_bytes: 2000, max_lookups: 6}
27 + - {method: A7, codec: zstd}
28 + - {method: A8, budget_tokens: 512, access: blind}
29 + - {method: B0}
30 + - {method: B1}
31 + - {method: B2, bits: 8}
32 + - {method: B2, bits: 4}
33 + - {method: B2, bits: 2, group_size: 32}
34 + - {method: B3, keep_frac: 0.5}
35 + - {method: B3, keep_frac: 0.25}
36 + - {method: B3, keep_frac: 0.125}
37 + - {method: B4, keep_frac: 0.25}
38 + - {method: B5, keep_frac: 0.5, access: aware}
39 + - {method: B5, keep_frac: 0.25, access: aware}
40 + - {method: B5, keep_frac: 0.125, access: aware}
41 + - {method: B5, keep_frac: 0.25, access: blind}
42 + - {method: B6, keep_frac: 0.25, access: aware}
43 + - {method: B9, keep_frac: 0.25, bits: 8, access: aware, policy: attention}
modified configs/smoke.yaml +1 −1
@@ -6,7 +6,7 @@ limit_docs_per_family: 1
6 6 max_questions_per_doc: 4
7 7 model_pairs:
8 8 - {sender: qwen3b, receiver: qwen3b}
9 −generation: {max_tokens: 48}
9 +generation: {max_tokens: 48, per_family: {exec: 320}}
10 10 seeds: [0]
11 11 conditions:
12 12 - {method: A0}
modified data/generated/docs/exec/exec_00001_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00001_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 47.53, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 4.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 25, \"unit_price\": 71.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 9.99, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 50.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 26.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8073,
7 − "document_hash": "ddfe7fcc3baca3e97144b66188f285a0e8a83018d1ca41b7f37cb80dede62dcc",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 47.53, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 4.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 25, \"unit_price\": 71.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 9.99, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 50.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 26.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7894,
7 + "document_hash": "72a702cc7076fb313f97772498675d83532bf6ce59f1d3cbf3bb86fa387dc0e6",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00002_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00002_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 5.2, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 14.07, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 51.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 37.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 67.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8001,
7 − "document_hash": "7918e3cdcab2a8bfae77558b7b523f4cea70cdc52435393d98d0c7b36de29e60",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 5.2, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 14.07, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 51.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 37.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 67.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7819,
7 + "document_hash": "7ec85eb17fda1c57c2a77a31dc79bf35b3c62974f6f1905e409880cbe869b6f5",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00003_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00003_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 35.12, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 47.96, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 27.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 8.05, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 37.32, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7972,
7 − "document_hash": "8ab845867bddfdcb5cf63b1319a6f29794cc9dddea45cd92bca3bc94fe023da7",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 35.12, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 47.96, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 27.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 8.05, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 37.32, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7793,
7 + "document_hash": "f7d729b32469d142625681408a1dd1aa3d1d99a7c07adb645c9f40d7550e8695",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00004_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00004_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 66.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 75.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 70.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 56.79, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 16, \"unit_price\": 23.1, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 64.58, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 8037,
7 − "document_hash": "08c5cdfaa7a0742a094ee4726d97a32a89395c354270406bbcf78eff4fdbe8b2",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 66.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 75.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 70.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 56.79, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 16, \"unit_price\": 23.1, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 64.58, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7855,
7 + "document_hash": "aeca575de77be2cf25f1c9f8e053485ad918ef4b3468680a4ef6bfa3b8cb781b",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00005_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00005_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 9.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 9.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 26.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 12.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 4, \"unit_price\": 49.06, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 38.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 8031,
7 − "document_hash": "f885d791d02ed1adb010c9c27804b5c38facf8fd2a09b9b1b6f9cacb7b0acc81",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 9.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 9.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 26.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 12.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 4, \"unit_price\": 49.06, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 38.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7852,
7 + "document_hash": "c5e26772985f1897f27e3729a56bdf95e08ba4718b928a1480572409c422c83f",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00006_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00006_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 51.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 16.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 75.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 55.85, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 45.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 32.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8049,
7 − "document_hash": "2a1bb6d86c77861b166e281bb1deac9a658ed4383a7ade93e9698e33861de0e5",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 51.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 16.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 75.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 55.85, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 45.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 32.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7867,
7 + "document_hash": "283461ef1dfaf2225620026d081a07116ec5fa2173ffa118704c0cf1d53a2eac",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00007_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00007_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 15.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 21.54, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 27.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 48.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 59.99, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 79.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 9}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 7826,
7 − "document_hash": "fb0150ce17a4b0ade1601cf6082b82e83f67db83a57c177064777d028b33d97c",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 15.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 21.54, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 27.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 48.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 59.99, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 79.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 9}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7644,
7 + "document_hash": "961b89accd7b0332a165c291bb3c527fd00bea183807cff965326b5f758f1b7f",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00008_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00008_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 27.18, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 55.08, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 28.44, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 59.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 79.85, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 26.43, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 8052,
7 − "document_hash": "04f896fa681f3ae7cbfc7efa30fd96c43be9d9b703dab76457ee8043483b3612",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 27.18, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 55.08, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 28.44, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 59.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 79.85, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 26.43, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7873,
7 + "document_hash": "4ad9a1fbe7db8f523841d367742505a5c92581010a15fdabb2bb6b6d77afbaa4",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00009_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00009_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 10.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 5.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 46.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 44.75, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 42.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8097,
7 − "document_hash": "db44cbab6b33239dddaa205976960167be792c1b1025bd39a4c4d6cf0f6ab7d7",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 10.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 5.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 46.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 44.75, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 42.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7916,
7 + "document_hash": "7c7609e3e3780713446d7ec4cbe48dba3ad69a1bf6d96393eb7e34fe13581d81",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00010_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00010_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 25.77, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 67.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 78.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 47.19, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 61.6, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 64.37, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 8004,
7 − "document_hash": "232fe52471a905a3ae14b591781c17dfa57a2d115258bfc1f8a1fce231b705ad",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 25.77, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 67.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 78.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 47.19, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 61.6, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 64.37, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7825,
7 + "document_hash": "2951e556f1045edd4bb9926de1dac81793260ddcd823e8c6abe38566b4cbcc07",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00011_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00011_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 66.05, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 28.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 44.45, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 7, \"unit_price\": 66.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 6.84, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 64.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8135,
7 − "document_hash": "f046c8292307adf97cae067d41f5a7bcce9d2a48bc8dd143e0f11e89a1c3a4c2",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 66.05, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 28.06, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 44.45, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 7, \"unit_price\": 66.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 6.84, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 64.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7953,
7 + "document_hash": "6b5b870bbcdf744c667890cf6185717b10f87e45ad7a6ccada501987d53f5e6d",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00012_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00012_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 45.84, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 28.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 22.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 52.18, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 46.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 7906,
7 − "document_hash": "2befcba5bfd5b509a9a6ec5d9334e2e384a05b6733ac02d80222302d3ca6e6ef",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 6, \"unit_price\": 45.84, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 28.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 22.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 52.18, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 46.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7724,
7 + "document_hash": "8991dd153dfd83b4a1aea0eb4340aabe4370ac72ef16427f5f67894ac175edde",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00013_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00013_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 19.02, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 31.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 6.76, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 50.8, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 70.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 29.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8051,
7 − "document_hash": "d006207e385cff8703be29071faf5845de84d05d8a677e40dda4983739115d29",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 19.02, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 31.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 6.76, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 50.8, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 70.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 29.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7870,
7 + "document_hash": "ee93ee22294a4463cb6cf9ca3d5ab7ea7fd2a991b2b55448f089c2290e51a174",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00014_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00014_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 20.49, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 61.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 28.29, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 53.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 5.5, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 60.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 8089,
7 − "document_hash": "e6ac9234df3fc7dcec86770e041bf75b576d1c65cf6fe32717891429f5461066",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 20.49, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 61.62, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 28.29, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 53.03, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 5.5, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 60.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7910,
7 + "document_hash": "b25f9d4fb9c997b0cb6e19983b48490097f6592b8d17e66f305f5a1b3e8a8443",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00015_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00015_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 44.28, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 77.11, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 37.27, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 24.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 49.63, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8041,
7 − "document_hash": "7bd733e903db470812271aa82f1cd9c7111ace0dcd257ace35b07e660d045be7",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 44.28, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 77.11, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 13.36, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 37.27, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 24.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 49.63, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7859,
7 + "document_hash": "7334bc5b260e11b2b0cdf9db05839cac2f397ae0e44db4e8bba7559dc94036bb",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00016_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00016_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 16.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 13.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 31.16, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 76.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 43.73, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 41.2, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8045,
7 − "document_hash": "b232fee620440e55facf033b892741387c97661b269dc2371313fb0f5934ff69",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 16.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 13.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 31.16, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 76.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 37, \"unit_price\": 43.73, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 41.2, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7864,
7 + "document_hash": "6a5268852caa8dc3015fd42e678e912f463ba061ac417b09de9ac6ed45e85d7f",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00017_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00017_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 71.87, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 57.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 63.43, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 39, \"unit_price\": 73.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 41.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 17, \"unit_price\": 36.3, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8053,
7 − "document_hash": "bce532bd3702b32e6eb3620ef3ea7d860a695f22bb90b8e995836a7937c77f52",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 71.87, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 57.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 63.43, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 39, \"unit_price\": 73.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 41.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 17, \"unit_price\": 36.3, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7872,
7 + "document_hash": "f2d06422bd98b400b7401ce4c5ae2939aeb43f51482e84cba0c0e3c0845c71a5",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00018_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00018_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 62.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 68.54, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 63.9, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 71.94, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 9.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 26.85, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 8169,
7 − "document_hash": "6f0951d7af338593059efbe431f0b7ce5bf9ae652e3dfb9eedafa7e192b80c01",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 62.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 68.54, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 63.9, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 71.94, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 9.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 26.85, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7990,
7 + "document_hash": "f219110fda4e5d20fabbe592248da3e78f61f411dbc16b9412d9b8d692eda870",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00019_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00019_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 55.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 5.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 35.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 18.49, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 38.49, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 40, \"unit_price\": 46.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8031,
7 − "document_hash": "733853d52a559dfc67d0cdab02de9e21c2aa09e0b25702d2cadfb38c20ab2159",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 55.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 5.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 35.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 18.49, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 38.49, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 40, \"unit_price\": 46.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7849,
7 + "document_hash": "232819b55ebc60215845a14aaf01c04a94dc7902933c03a47546ec41ee16502f",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00020_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00020_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 77.87, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 51.72, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 67.82, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 22.1, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 49.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 8080,
7 − "document_hash": "1a7bfffc3347254640c8b76b9b5f0ff368a758543995fa06006e28ea43d8eb71",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 77.87, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 51.72, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 19, \"unit_price\": 11.75, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 67.82, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 22.1, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 49.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7901,
7 + "document_hash": "862136f6ccb7f77097086ceddd84945e55b7df17401a63b0ce306740969966e5",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00101_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00101_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 53.66, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 17.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 66.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 33.18, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 47.37, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 22.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8125,
7 − "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 53.66, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 17.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 66.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 33.18, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 47.37, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 22.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7944,
7 + "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00102_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00102_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 68.35, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 65.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 17, \"unit_price\": 8.56, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 36, \"unit_price\": 48.5, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 22, \"unit_price\": 76.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 51.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8089,
7 − "document_hash": "5535ac3109616f7d46dff5417535ca232754aee2c0759e072d552e465f004d46",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 68.35, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 65.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 17, \"unit_price\": 8.56, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 36, \"unit_price\": 48.5, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 22, \"unit_price\": 76.92, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 51.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7910,
7 + "document_hash": "61b6c7bb86261a26bf000996edd5f9326f7f77ca1c37085fd999cddf1bff2396",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00103_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00103_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 24.8, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 13.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 26.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 10.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 34.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 17.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8038,
7 − "document_hash": "a7875d1e95742c6246c4ec3862b9797a4269a2b607e47869f5014d7dc29139e0",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 24.8, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 13.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 26.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 10.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 34.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 17.34, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7856,
7 + "document_hash": "596ef1cd6d37fdac729520a738ed129e4ffaf619d372ad869d8188a0a5cac1ef",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00104_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00104_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 26.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 59.29, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 50.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 74.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 40.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8014,
7 − "document_hash": "bce3091895fc315bf981ac1272a7da1f00b0f64819f1ecfa2a6dd8a3d23f2207",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 26.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 59.29, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 50.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 74.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 40.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7835,
7 + "document_hash": "6b4aadaf1b71672d6d8aed5a412c7aa490e3e0eb1c92e5df6db1cc9554f83483",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00105_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00105_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 44.66, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 12.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 71.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 69.08, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 71.78, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 17.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8046,
7 − "document_hash": "fe4b8857bf1774be1123bec85a5743da5c3551d53125ddf128efde401f3990c2",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 44.66, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 12.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 71.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 69.08, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 71.78, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 17.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7867,
7 + "document_hash": "a59face90fe592d3d8a9b375aafefb775631dd798a60a424f981cceae1ec2996",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00106_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00106_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 66.67, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 69.05, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 67.57, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 49.2, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 40.78, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 31, \"unit_price\": 40.56, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 7943,
7 − "document_hash": "c3652bfa1632f0cc62d6512131ef519100143881225c8b4ee572c1876e7c6c7f",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 66.67, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 69.05, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 9, \"unit_price\": 67.57, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 49.2, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 40.78, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 31, \"unit_price\": 40.56, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7761,
7 + "document_hash": "a78f55af374fd04eb589b193a9eca2f84cf04bcdb3fa60aaeb29fbe358c79464",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00107_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00107_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.74, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 4.67, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 67.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 10.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 61.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 16.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 7930,
7 − "document_hash": "983dbcacdfae2acf0316690a7fe50cfa2523ae41beffd9de42356f2ebd3d1e79",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.74, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 4.67, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 67.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 10.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 61.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 16.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7751,
7 + "document_hash": "83ea6e4f6c5d3a6eed7a921a930819d760ffb3dac7c22761dc9b4c28084904cc",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00108_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00108_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 59.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 47.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 24.32, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 53.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 33.13, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 58.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 8038,
7 − "document_hash": "7ebb05d3714e237d074dfc9a69c5b83e99932f9041b5d4fe15c8fc7224ba7078",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 16, \"unit_price\": 59.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 47.72, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 24.32, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 53.1, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 33.13, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 58.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7859,
7 + "document_hash": "b9ccaaf72598ea6e12a10942f2697dd01ca891115b652043a2f6ce4cb68570cc",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00109_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00109_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 16.34, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 77.14, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 21.7, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 65.34, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 36.24, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.17, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8073,
7 − "document_hash": "680dab47e67683cb351941eea647197cb2b2ea472ff116582019ace0c543d461",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 16.34, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 14, \"unit_price\": 77.14, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 21.7, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 65.34, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 36.24, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.17, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7894,
7 + "document_hash": "d53ada6a135b4827dcacb2ebec142b5000ad21961fb0cd199fc7c609424d4d5a",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00110_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00110_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 48.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.65, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 52.8, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 30.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 18.59, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 26.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 8019,
7 − "document_hash": "f1699da28903736ad58ec0640a277bb2eac030c80c49c22c027c273374df86fa",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 48.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 37.65, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 52.8, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 25, \"unit_price\": 30.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 17, \"unit_price\": 18.59, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 26.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7840,
7 + "document_hash": "789f1f1a750d514137a988d21a0807cc30c8548da830f2eeef66854fc179fcf5",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00111_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00111_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 74.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 62.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 46.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 49.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 62.57, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 38.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 7942,
7 − "document_hash": "e65e9534b60b15cdaadf465396c6385868f5a0ed4be5ba04c458937c2eff43ae",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 74.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 2, \"unit_price\": 62.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 46.48, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 49.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 62.57, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 38.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7763,
7 + "document_hash": "9f3d9d2caa1830b0bf0b983179f4de3251ec9bb8648c710bf43f0fb6c531ef85",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00112_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00112_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"gamma\", \"units\": 4, \"unit_price\": 38.52, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 13, \"unit_price\": 23.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 18.69, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 55.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 19.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 61.42, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 − "char_len": 7996,
7 − "document_hash": "4a8db801f5e7f6b2108618514f90c37fbdbb61e902c4441c0d68548a5792da05",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"gamma\", \"units\": 4, \"unit_price\": 38.52, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 13, \"unit_price\": 23.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 18.69, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 55.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 19.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 61.42, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 + "char_len": 7814,
7 + "document_hash": "182cf1cbb5939b7e1ca9170096aa77a9cc3c2c4f8efb3195e8b4ad19b77a3f46",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00113_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00113_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 39.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 5.13, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 20.94, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 3.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 42.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 3, \"unit_price\": 57.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 8055,
7 − "document_hash": "7535b4d324a3e28b457e40fe41d18d52e75a9ed0271b411285e522c6c8c8200e",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 39.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 5.13, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 20.94, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 3.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 42.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 3, \"unit_price\": 57.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7874,
7 + "document_hash": "c89720bd112bca637c8fa3b4a98f7cc41a384be468eee47f5c44607ee6fc46f1",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00114_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00114_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 39.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 65.95, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 38, \"unit_price\": 38.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.5, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.46, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 10.31, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 6}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8042,
7 − "document_hash": "e1faab8175075b8cf98d2e5e16181db3f5a8f103ed89d35efee7ba4684968699",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 39.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 65.95, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 38, \"unit_price\": 38.36, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.5, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 23, \"unit_price\": 30.46, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 10.31, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 6}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7863,
7 + "document_hash": "86a377fe8f0559479ca21128972c50d14bd9e19d38d2bf49f763d402259c0aad",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00115_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00115_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 62.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 35, \"unit_price\": 64.7, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 29.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 17.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 53.93, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6 − "char_len": 8127,
7 − "document_hash": "4d6a9aa137c992493ef42f6758dbb35b873a4a392b00eae7152d6e45b68a3c1b",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 62.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 35, \"unit_price\": 64.7, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 2, \"unit_price\": 29.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 17.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 53.93, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6 + "char_len": 7946,
7 + "document_hash": "33ee1d20ea5713048e1335d3786ba0b10b3a5998a16200f09823d0c4cf0dcbe1",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00116_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00116_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 40.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 3, \"unit_price\": 67.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 21.59, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 21.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 30.51, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 66.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 7941,
7 − "document_hash": "2b6100ff4b04865ebf622b9749757f3b32215faa2e32f12d63be71b0c18fc7be",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 27, \"unit_price\": 40.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 3, \"unit_price\": 67.68, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 21.59, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 21.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 30.51, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 11, \"unit_price\": 66.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7762,
7 + "document_hash": "379dcf9e9ada65463c8d73e2cbb54ff33c8c46ea3154539a1d6df25bab2a0a0e",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00117_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00117_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 34.17, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 39.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 37.26, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 13.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 4.12, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 1, \"unit_price\": 4.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 8064,
7 − "document_hash": "6e53d886a35d8e51c98908f1e40e9314353c62a19e3197603509fd2a59c6bbf3",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 34.17, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 39.09, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 37.26, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 13.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 19, \"unit_price\": 4.12, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 1, \"unit_price\": 4.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7883,
7 + "document_hash": "34a1cc9614d0c2d2f9766b2968ae3fa3eb2b6cc1a372750650ae7f7359325896",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00118_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00118_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 68.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 35.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 65.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 2, \"unit_price\": 53.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 48.57, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 8050,
7 − "document_hash": "5353584056feb451c43bd2199aa87e2f7064bbfb9485b6eb1f6a1387dddac0f6",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 68.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 35.36, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 15, \"unit_price\": 65.25, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 8, \"unit_price\": 18.19, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 2, \"unit_price\": 53.23, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 48.57, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7868,
7 + "document_hash": "58d2108f993d9baafb67222dc4bd97895639cb6c2410e675e3ed2c4233c8fee4",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00119_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00119_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.28, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 28.27, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 56.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 58.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 21, \"unit_price\": 7.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 10.91, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 7901,
7 − "document_hash": "f62c9b5ecd91caa0cf95af49bf6d40dd2bc3885255008434f482e56b7a2a2fae",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 38.28, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 28.27, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 56.96, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 58.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 21, \"unit_price\": 7.09, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 20, \"unit_price\": 10.91, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7722,
7 + "document_hash": "98f121e7a33ff30acfcaf7a0346bbc04cc9f508e092e316490793765a8e0ab09",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00120_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00120_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 26, \"unit_price\": 76.39, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.78, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 55.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 48.6, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 31.92, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 33.32, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7992,
7 − "document_hash": "5b1b1089a793b0953579b8ce004efe99282ffd8a6eb17b16c3ce50273c121aad",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 26, \"unit_price\": 76.39, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 2, \"unit_price\": 45.78, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 55.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 48.6, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 31.92, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 33.32, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 5}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7811,
7 + "document_hash": "c5be33e5009123ba2c251884304b839ae6fc5163f0745c302cffbabf618b40cb",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00121_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00121_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 48.97, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 58.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 78.34, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 65.31, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 52.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 37, \"unit_price\": 9.99, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 − "char_len": 8033,
7 − "document_hash": "9064e893e331010c27db6e1229c2af7da83ed7fb3680e079fc5c1a9ee7b2d73a",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 48.97, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 58.75, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 27, \"unit_price\": 78.34, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 65.31, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 52.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 37, \"unit_price\": 9.99, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 + "char_len": 7854,
7 + "document_hash": "1645df3901b5bd61ea8c6790e15a695b1144eef69bc075121ffd0c83e6fc13af",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00122_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00122_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 43.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 74.34, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 31.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 12.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 24.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 38, \"unit_price\": 69.36, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8139,
7 − "document_hash": "8641f3a09b033f2a216948ea197682c280cda7a9559b7102473da5f152a73eb3",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 43.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 74.34, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 29, \"unit_price\": 31.64, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 12.23, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 24.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 38, \"unit_price\": 69.36, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 12}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7960,
7 + "document_hash": "bd26e436a86876d04961cc142588671db4ead753f838933bde6907da5354f70e",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00123_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00123_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 44.11, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 43.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 27.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 15, \"unit_price\": 68.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 49.94, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 57.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 8069,
7 − "document_hash": "3a8a85801b1200bb0a18063e20eb67097e3e9c34d009894a5a61963f7145608b",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 44.11, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 43.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 27.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 15, \"unit_price\": 68.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 1, \"unit_price\": 49.94, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 57.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7890,
7 + "document_hash": "6c868c5916de1b984d70dc327ee458814726f8ea6a1fad99f8c2fb35e7e77b7f",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00124_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00124_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 15.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 16.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 32.63, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 22.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 53.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 66.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 7919,
7 − "document_hash": "adfa94aa242540ae900f58c2db73dace6e0bb6ec243a71137685b2d527e54fdb",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 24, \"unit_price\": 15.43, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 16.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 32.63, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 25, \"unit_price\": 22.72, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 53.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 66.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7740,
7 + "document_hash": "a285cd3f7f5bbfcaa508697bf261341dc847483a97e354b95faf09bc23bb626f",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00125_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00125_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 74.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 79.31, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 24.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 61.06, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 48.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 73.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 7915,
7 − "document_hash": "144a4af055228fc4045ad15b1ca9ba8229c8e5563cc4c4cf9ad94203a0072dd9",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 34, \"unit_price\": 74.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 79.31, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 24.62, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 61.06, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 48.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 73.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7733,
7 + "document_hash": "386fcaff7daca426267d33af6f30657f21e4863b66d5a71d38d0b78a334eb069",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00126_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00126_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 14.21, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 40.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 66.39, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 5, \"unit_price\": 58.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 35.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 51.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 7944,
7 − "document_hash": "a6bed1ca1c6bee001ded092712010d4dc8d3ec10d0d1f85432e8844be28b9359",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 14.21, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 40.29, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 66.39, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 5, \"unit_price\": 58.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 29, \"unit_price\": 35.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 51.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7763,
7 + "document_hash": "0729452f4b76415c15bdd0075e98c4db16464515a675ea3d8e3611adb3c8a3e9",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00127_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00127_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 4.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 57.56, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 48.4, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 7.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 49.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.89, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 7953,
7 − "document_hash": "01e9b8b715cbfdd0d66c26d571044c18f9a6d9d2c5146b834291716879cf02d2",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 4.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 57.56, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 48.4, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 7.69, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 4, \"unit_price\": 49.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 48.89, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7774,
7 + "document_hash": "bb6e6467042a1b7809f3ea38b83ea82e5554ca36b70042023f682efd5bae752a",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00128_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00128_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 33.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 74.94, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 71.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 69.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 38.14, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 7945,
7 − "document_hash": "e702731f232f4b91d0513add4b301fd1ef0382eee35df22a98cef7eb727fafca",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 33.84, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 3, \"unit_price\": 74.94, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 29.12, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 71.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 69.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 22, \"unit_price\": 38.14, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7763,
7 + "document_hash": "ea7fe68afa60ebc43cd0a501c243c72cd3b993bfaef6e389f12308e1967e3471",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00129_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00129_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 31.69, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 44.06, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 28.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 18.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 32.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 10.85, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 − "char_len": 7973,
7 − "document_hash": "e508132470ec83c8745c1899d6c954b865130cad33382718fa48d35d6bea0ec8",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 31.69, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 44.06, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 17, \"unit_price\": 28.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 18.96, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 32.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 10.85, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 + "char_len": 7794,
7 + "document_hash": "46f3b799f18b08ee443481845e8695eff10ac2f2b23c2b177eafce3b42e2ea40",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00130_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00130_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 4.14, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 3.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 18.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 23.95, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 17.39, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 71.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8072,
7 − "document_hash": "c65f14fbcb1d4d0049a6e641be2826817637866851032f95c4cc1943f87ff809",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 4.14, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 3.41, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 18.56, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 4, \"unit_price\": 23.95, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 17.39, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 30, \"unit_price\": 71.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7893,
7 + "document_hash": "c415053d25119b89bd769d02be02c390efebcd68b992b946ddf043ad29041f86",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00201_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00201_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 5.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 15.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 54.9, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 7.38, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 33, \"unit_price\": 19.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 43.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 8073,
7 − "document_hash": "876f090ca1801217ec4295964e7d9ce5b46b9fc96b04db7ca463d6cc8ed74978",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"beta\", \"units\": 18, \"unit_price\": 5.04, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 10, \"unit_price\": 15.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 54.9, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 39, \"unit_price\": 7.38, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 33, \"unit_price\": 19.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 43.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7892,
7 + "document_hash": "065b6c18bbd16f16126924a54946ca4bf95c5863e57b14f33e4c9d88568f8549",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00202_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00202_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 68.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 71.74, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 31.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 43.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 5, \"unit_price\": 75.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 76.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 7844,
7 − "document_hash": "660fc77600d7958889cef9b34f9c9a1a8c2d96ef38db054d180f225ebe3d4a42",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 68.95, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 71.74, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 6, \"unit_price\": 31.62, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 32, \"unit_price\": 43.27, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 5, \"unit_price\": 75.21, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 76.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7665,
7 + "document_hash": "ec13194237af7efe7765a21f1d8303ab8ac449271eb48e3395906648c770bab8",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00203_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00203_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 73.68, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 40.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 27.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 78.27, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 49.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 53.29, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 − "char_len": 7967,
7 − "document_hash": "f26e049133c3f806d2045fa255ba4e6c2594f5ecbe872414b75faaf52ceb5033",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 73.68, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 40.46, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 21, \"unit_price\": 27.47, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 8, \"unit_price\": 78.27, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 49.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 30, \"unit_price\": 53.29, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 + "char_len": 7788,
7 + "document_hash": "39eee37c7ba5fe1b6d8fce9eef4c00e10a33d8ca049d12ae01b9521d4ac60863",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00204_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00204_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 22.64, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 40, \"unit_price\": 49.16, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 47.27, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 36, \"unit_price\": 38.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 29.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 22.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 8100,
7 − "document_hash": "7cc296695d97fdf8fb07de5772744af9c3a79a52f20ee1e3410187cac4dd94b2",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 14, \"unit_price\": 22.64, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 40, \"unit_price\": 49.16, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 47.27, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 36, \"unit_price\": 38.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 29.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 26, \"unit_price\": 22.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7921,
7 + "document_hash": "0aaf1a095160aa21c4485db3b1b171e86d9d41b904ac2a2ea102cc13df19cd22",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00205_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00205_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 21.72, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 33.96, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 7, \"unit_price\": 20.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 24.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 29.74, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 1, \"unit_price\": 9.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 8061,
7 − "document_hash": "1f9b534774ca095126415c70bc29f4400d55f083ec020a658e6ee1263a76072f",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 11, \"unit_price\": 21.72, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 4, \"unit_price\": 33.96, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 7, \"unit_price\": 20.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 40, \"unit_price\": 24.33, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 29.74, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 1, \"unit_price\": 9.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7882,
7 + "document_hash": "c82f149ddbd77be80773d103825621fb187f06bb60d4fd570c73c2b3cd898ecb",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00206_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00206_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 41.55, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 10.75, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 79.56, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 44.39, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 12.61, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 35.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 − "char_len": 7908,
7 − "document_hash": "71e5bf98eb246c53babd99cdd8ec1212e594a1af3ddd2a6da69bac721ad65ddd",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 41.55, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 10.75, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 79.56, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 5, \"unit_price\": 44.39, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 12.61, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 35.76, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 + "char_len": 7727,
7 + "document_hash": "e391c9d757bc03677c664222c3ff3f1bbab881d7f566c0072b184e0b90503339",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00207_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00207_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 61.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 57.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 30.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 25.39, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 54.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 66.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 − "char_len": 8035,
7 − "document_hash": "2273befa734c54f0d780db1126ec3ce69ce8f9f829e57586a614786258681ba3",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 61.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 57.71, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 30.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 26, \"unit_price\": 25.39, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 32, \"unit_price\": 54.7, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 66.46, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 + "char_len": 7856,
7 + "document_hash": "7d024469b5d38776a899aa870f5af7e146953a19f1adc1d91f634cda62aa4c14",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00208_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00208_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 34.08, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 10, \"unit_price\": 62.88, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 63.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 23.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 78.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 32, \"unit_price\": 71.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 8036,
7 − "document_hash": "e22e990c3c20d7feee2f70e993961e465ac68d3a867c3547dc0566739847c2d8",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 33, \"unit_price\": 34.08, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 10, \"unit_price\": 62.88, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 27, \"unit_price\": 63.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 23.2, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 35, \"unit_price\": 78.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 32, \"unit_price\": 71.98, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7854,
7 + "document_hash": "8528b3fafcf58b9daa88ffa9d7b9b25fb6e225f9bb767849744c05510a5ace37",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00209_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00209_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 42.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 72.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 23.31, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 63.77, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 32.46, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 52.71, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8044,
7 − "document_hash": "bd895760d266f605848302775beaceb129633108ddc74dcbe83a8642eecb0ea3",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 42.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 72.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 38, \"unit_price\": 23.31, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 63.77, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 32.46, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 52.71, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7865,
7 + "document_hash": "27a0f4409b3611173adf72865b52c6beef014bce0282fd0498be50deeed192c9",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00210_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00210_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 64.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 76.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 77.13, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 33.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 77.57, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 19.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7990,
7 − "document_hash": "2678fb8184c3c22ce8bfa861af1a68ef2704dd84cb88718d63c1b668adaf1f69",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 15, \"unit_price\": 64.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 26, \"unit_price\": 76.61, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 77.13, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 28, \"unit_price\": 33.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 77.57, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 19.87, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7811,
7 + "document_hash": "ffa82307f9b4986f3fc473b8b7df1e4f8f0e1c25d3b6e36cdcf4114a763427c5",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00211_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00211_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 44.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 18.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 19.62, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 61.32, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 64.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 15.18, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 8091,
7 − "document_hash": "5ad4fd06e4149025eddafab6fbf5c707e06c1f7e47f3cbaec839f52cd6744e38",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 44.73, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 18.17, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 19.62, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 34, \"unit_price\": 61.32, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 11, \"unit_price\": 64.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 15.18, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7909,
7 + "document_hash": "95faa51b0302db54273e9b4469b6eae92c251463238fdad494d437435c293548",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00212_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00212_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 38.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 5.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 68.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 31.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 68.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 17.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8061,
7 − "document_hash": "8a171bf76e4dc9c8921e9b2a4370fc5385ceb9695d62d19fc693110c84df06f0",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"delta\", \"units\": 36, \"unit_price\": 38.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 5.47, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 68.81, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 5, \"unit_price\": 31.92, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 68.1, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 22, \"unit_price\": 17.46, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7882,
7 + "document_hash": "2be5dc8496484a99fc11a00be474f4f7f139a8f491a2ebf3fff0db8e7804dd82",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00213_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00213_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 6.93, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 72.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 79.31, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 20.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 16.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 59.43, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 7876,
7 − "document_hash": "55fc33fe81ccfe25e3ebdcdb70ee135b990fcf94d02bfb56e0116982ffd6df6f",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 21, \"unit_price\": 6.93, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 40, \"unit_price\": 72.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 79.31, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 20.15, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 16.11, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 35, \"unit_price\": 59.43, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 10}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7695,
7 + "document_hash": "3c65c42333c25f1f3f90331747f08fafce263c8faa11a56d3a68bc58b2217327",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00214_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00214_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 62.71, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 20, \"unit_price\": 4.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 37.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 54.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 24.23, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 14, \"unit_price\": 46.97, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 7997,
7 − "document_hash": "0be81f6d80f5eb03ba862cbc94c8bf20b6ab1196483c947401585e7178fd458d",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 20, \"unit_price\": 62.71, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 20, \"unit_price\": 4.15, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 20, \"unit_price\": 37.58, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 35, \"unit_price\": 54.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 24.23, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 14, \"unit_price\": 46.97, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7815,
7 + "document_hash": "ac0a3ee69ec7da626616f6ffb5e443a4f26a3d7857caaa20f1d4ebfa5355d28b",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00215_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00215_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 33.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 2.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 34.09, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 66.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 32.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 39.54, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6 − "char_len": 8073,
7 − "document_hash": "37b95a047621c2a78beaff095b27647064c994bf6ede6ef6b3e91f8296b70eff",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 33.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 2.58, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 34.09, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 66.24, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 32.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 25, \"unit_price\": 39.54, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6 + "char_len": 7894,
7 + "document_hash": "a4094c70181359b8442ead384be9497e3fb0c46a4d9266cce07ca77bd9e26e2b",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00216_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00216_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 19.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 57.35, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 56.48, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 70.24, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 66.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 33.67, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8017,
7 − "document_hash": "1deddf301b0b2291cb9bdc4213848eceb17ece12032a2e454bcceb4ca4174397",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 30, \"unit_price\": 19.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 3, \"unit_price\": 57.35, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 18, \"unit_price\": 56.48, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 70.24, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 66.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 35, \"unit_price\": 33.67, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7838,
7 + "document_hash": "1a218de32e6fe4bbe97927ea16330bda3b598ed66c1517824d94ff90f944e802",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00217_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00217_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 71.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 23.77, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 22, \"unit_price\": 39.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 40.72, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 36.45, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 51.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 7946,
7 − "document_hash": "7f71cb12d994e6fe2646a0f7b337c441f923868ec6b21c29533122fda4d87a73",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 22, \"unit_price\": 71.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 23.77, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 22, \"unit_price\": 39.11, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 40.72, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 6, \"unit_price\": 36.45, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"epsilon\", \"units\": 12, \"unit_price\": 51.53, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7764,
7 + "document_hash": "1223c7391b1da1aeae2d242c0e41de4639218cfe0c90ba9065f1cce42f0eedbd",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00218_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00218_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 69.57, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 11, \"unit_price\": 53.59, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 77.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 24, \"unit_price\": 22.35, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 79.4, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 20.83, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 7955,
7 − "document_hash": "2a77fd38142317b7e8868dfcaebb42248ddd462e9339268378a26e4b587eab3d",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 69.57, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 11, \"unit_price\": 53.59, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 77.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 24, \"unit_price\": 22.35, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 79.4, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 38, \"unit_price\": 20.83, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7776,
7 + "document_hash": "971d94c4e7b4ae02142ffb6eb92af9d2cc0f6e4d52e94a64c5132b510197ce8a",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00219_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00219_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 79.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 39, \"unit_price\": 77.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 27.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 6.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 50.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 13.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8119,
7 − "document_hash": "501d502ac1cd6920ddaeccbc4f0c925ddf4ca966cfb601ffd9d0547d97a741ed",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 34, \"unit_price\": 79.78, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 39, \"unit_price\": 77.53, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 27.42, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 6.47, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 50.95, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 10, \"unit_price\": 13.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7938,
7 + "document_hash": "390edd848add1a4cf011660e8ac9a6a6aa381c4508a8f68635f3217ce20979cb",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00220_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00220_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 67.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 37.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 44.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 27.9, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 7.8, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 61.13, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7931,
7 − "document_hash": "a7487c2eb12039fcc44c64460641febdd19eaf0e13d8b6b8f00d018ca564a8b6",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"beta\", \"units\": 16, \"unit_price\": 67.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 4, \"unit_price\": 37.2, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 31, \"unit_price\": 44.3, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 39, \"unit_price\": 27.9, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 28, \"unit_price\": 7.8, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 33, \"unit_price\": 61.13, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7750,
7 + "document_hash": "31eca9e7afc70ac5a795edc1d1529c531e0d028342802a61f4496c013e13bf45",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00221_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00221_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 67.75, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 38.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 52.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 29.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 7.64, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 39, \"unit_price\": 27.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 7820,
7 − "document_hash": "a986eec8e76aa658fd694272657b8e201d940be0671e7769a934ffe76ca4748a",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 67.75, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 38.86, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 29, \"unit_price\": 52.88, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 12, \"unit_price\": 29.86, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 36, \"unit_price\": 7.64, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 39, \"unit_price\": 27.73, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7638,
7 + "document_hash": "e228dfaa0d04173ee28b43513e5fc06914edf5d0c02b09bf0fc534531a583669",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00222_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00222_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 13.55, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 25.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 19.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 79.51, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 42.07, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 57.83, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 9}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8057,
7 − "document_hash": "fa31c738b8b80170524adedf048965eb739a7ebf7de9fcd4f4192d331dcc3346",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 32, \"unit_price\": 13.55, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 25.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 19.2, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 6, \"unit_price\": 79.51, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 42.07, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 57.83, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 9}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7878,
7 + "document_hash": "040260f25a42680fcc038386368506adaa004bc2592e75e3fbb5903b54cc19ab",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00223_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00223_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 19.44, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 15.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 51.74, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 11.86, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 33.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 44.89, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 7992,
7 − "document_hash": "9b2d718a88e7b451a6e6d433e81fa6c3862777588f40791fe888b2d7a44efa42",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 10, \"unit_price\": 19.44, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 25, \"unit_price\": 15.63, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 37, \"unit_price\": 51.74, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 24, \"unit_price\": 11.86, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 22, \"unit_price\": 33.09, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 29, \"unit_price\": 44.89, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 1}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7813,
7 + "document_hash": "7eef13486249d89cf05bda367ea9dae2ce2479b2236c1917b2f067c49a981619",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00224_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00224_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 40.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 74.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 67.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 44.81, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 79.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 60.0, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 7921,
7 − "document_hash": "526a53c484a99dde8d83b8fc83d103060128d6331659920d7688b8836de1ec53",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 40.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 10, \"unit_price\": 74.89, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 5, \"unit_price\": 67.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 24, \"unit_price\": 44.81, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 12}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 9, \"unit_price\": 79.23, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 12, \"unit_price\": 60.0, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 7}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7739,
7 + "document_hash": "3e642abb57841365b827053f2437a250dcb419539389820bac778d4596c064fd",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00225_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00225_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 5.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 25.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 22.38, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 3, \"unit_price\": 5.01, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 70.26, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 5.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 7998,
7 − "document_hash": "0d5d08bcef45a939605c15cb0d0b5b1ac3c2304c8358dae05fa7da75db07621f",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 34, \"unit_price\": 5.3, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 25.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 38, \"unit_price\": 22.38, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 3, \"unit_price\": 5.01, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 19, \"unit_price\": 70.26, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 8, \"unit_price\": 5.79, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7819,
7 + "document_hash": "54918c16a4ae3706a87d182112fca2a39c82c77e3a60fd25a8b47b958f2a6b21",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00226_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00226_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 13.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 77.55, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 40.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 65.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 43.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 50.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7972,
7 − "document_hash": "32ca9297d301e1956917680ad370eda2add0304f9742c49a2023cfa3f2a82bd2",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 13.55, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 23, \"unit_price\": 77.55, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 24, \"unit_price\": 40.4, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 36, \"unit_price\": 65.14, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 8, \"unit_price\": 43.47, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 26, \"unit_price\": 50.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7793,
7 + "document_hash": "7a73067df93b17aba1a608bce48f7a6ff16d480af8d23713ec889405f279fc7d",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00227_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00227_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 60.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 16, \"unit_price\": 74.12, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 66.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 49.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 14.6, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 54.41, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 − "char_len": 8074,
7 − "document_hash": "0297d0253d8ecb7ba91156d4221aff1d54ba5efaed7fe7664734e63d44d31bd0",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 30, \"unit_price\": 60.69, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 16, \"unit_price\": 74.12, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 38, \"unit_price\": 66.15, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 49.85, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Centre\", \"product\": \"beta\", \"units\": 6, \"unit_price\": 14.6, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 1, \"unit_price\": 54.41, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 9}]\n\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.",
6 + "char_len": 7895,
7 + "document_hash": "6afe19e10d57be0082db797187045b004680fa22a13a56a7b8f7e2a51b40597d",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00228_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00228_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 76.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 29, \"unit_price\": 33.94, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 14, \"unit_price\": 14.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 4.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 52.45, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 77.48, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7994,
7 − "document_hash": "7bd8eb7ac7409f89d5f44f1f5001fd0660aece03d12d92588e821c86e0d1f69b",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 76.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Centre\", \"product\": \"epsilon\", \"units\": 29, \"unit_price\": 33.94, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 2}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 14, \"unit_price\": 14.45, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 6, \"unit_price\": 4.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 52.45, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 77.48, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 11}]\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7815,
7 + "document_hash": "08d5c40fa24cccbcccaac14d408c241ab602604a712b727f12f570eb56887a74",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00229_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00229_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 76.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 78.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 41.6, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 3.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 31.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 7905,
7 − "document_hash": "488c5efd60f8e64b88269226071b8befe058a5e2442b99934157b38f5fc41a19",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 24, \"unit_price\": 76.49, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 13.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 78.0, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 41.6, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 3.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 27, \"unit_price\": 31.65, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7723,
7 + "document_hash": "b6a34443a51bb95703036fc2467d0c50ad248f99f3c11c0a35d6bc7fcf0ad481",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00230_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00230_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 34.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 36.88, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 42.84, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 25.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 20.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 69.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6 − "char_len": 8012,
7 − "document_hash": "2e09e5c09991cb1b7c5c288b9a49bb058042a17dc7fd32cde98e2b705477598c",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 34.41, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 36.88, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 33, \"unit_price\": 42.84, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 1}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 20, \"unit_price\": 25.91, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 12, \"unit_price\": 20.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 69.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.",
6 + "char_len": 7830,
7 + "document_hash": "cbb9dba4a75d2570390205536d928042eae0341c49e7dcc11a2cea988c175ba9",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00231_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00231_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 34, \"unit_price\": 42.27, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 66.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 77.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 28, \"unit_price\": 73.35, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 48.71, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 4.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8077,
7 − "document_hash": "9071b939e284c02834a84c70b2af1783b2a26d6217aa2184b3224f0c27400eaf",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 34, \"unit_price\": 42.27, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 21, \"unit_price\": 66.28, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 77.97, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 28, \"unit_price\": 73.35, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"gamma\", \"units\": 18, \"unit_price\": 48.71, \"discount_pct\": 15, \"returned\": true, \"shipped_days\": 7}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 16, \"unit_price\": 4.12, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 1}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7895,
7 + "document_hash": "b599ec4f6ea744baef17b36d05ab828f105675eeebf81a780befd49f389ec9d3",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00232_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00232_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 25.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 75.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 65.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 7, \"unit_price\": 59.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 11, \"unit_price\": 48.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 30.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7990,
7 − "document_hash": "6bb55261f58b559a6304971baddf54cabc7de9843d30afe350a659e21fa02b73",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"alpha\", \"units\": 28, \"unit_price\": 25.31, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 3, \"unit_price\": 75.72, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 65.33, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 7, \"unit_price\": 59.04, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 11, \"unit_price\": 48.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 13, \"unit_price\": 30.99, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7811,
7 + "document_hash": "94e54bf28dac5a404fdf94451d3e632c18f63dff001cc758f89f5239d27f659b",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00233_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00233_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 40.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 16, \"unit_price\": 13.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 1, \"unit_price\": 30.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 77.63, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 21.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 7.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 − "char_len": 7909,
7 − "document_hash": "f88eb8610ee0ae9a0cc4c953e6f302bceb7e6a9f131c2e5f775a4026d048e997",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Est\", \"product\": \"zeta\", \"units\": 31, \"unit_price\": 40.36, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 16, \"unit_price\": 13.05, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 1, \"unit_price\": 30.93, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 77.63, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 12, \"unit_price\": 21.67, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"zeta\", \"units\": 9, \"unit_price\": 7.88, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 6}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.",
6 + "char_len": 7730,
7 + "document_hash": "c0f56e7b9b73df623955c8b6822884e4b53df031c58736213f498ed2b4d693dc",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00234_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00234_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 43.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 31.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 28, \"unit_price\": 16.65, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 40.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 33.7, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8006,
7 − "document_hash": "89b49129e4581b4b424c4986c040ee62a62ff6f482d53151d96cbcf944c97e02",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 43.38, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Ouest\", \"product\": \"gamma\", \"units\": 13, \"unit_price\": 31.73, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Ouest\", \"product\": \"beta\", \"units\": 28, \"unit_price\": 16.65, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 7}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 29, \"unit_price\": 40.15, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 33.7, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7827,
7 + "document_hash": "3522724b66617b60e55fa67efc23998ff528130352e7a960ee1e052d697670a5",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00235_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00235_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 21, \"unit_price\": 57.5, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 43.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 70.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 13, \"unit_price\": 4.78, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 3.2, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 1, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 − "char_len": 8079,
7 − "document_hash": "7870795b3e6c98bee59eb42d0e036ccc1d88fe696a6e7c94016c77fb0f3cefe4",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"zeta\", \"units\": 21, \"unit_price\": 57.5, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"delta\", \"units\": 21, \"unit_price\": 43.44, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 19, \"unit_price\": 70.07, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 9}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 13, \"unit_price\": 4.78, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 3.2, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Sud\", \"product\": \"beta\", \"units\": 1, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 5}]\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.",
6 + "char_len": 7900,
7 + "document_hash": "20bfca2f1ed7277720116b37988eb2bf9331505c61f7491ff7f131ed0b2d70e1",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00236_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00236_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 35.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 13.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 55.74, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 26.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 20.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 33.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 − "char_len": 8048,
7 − "document_hash": "dabe94079fad1aa1fef7653ff5583522be249aa0c33c58334ae1d261091d83f9",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 23, \"unit_price\": 35.91, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"zeta\", \"units\": 27, \"unit_price\": 13.03, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Centre\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 55.74, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 15, \"unit_price\": 26.26, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 12}, {\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 23, \"unit_price\": 20.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 11}, {\"region\": \"Centre\", \"product\": \"gamma\", \"units\": 15, \"unit_price\": 33.87, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.",
6 + "char_len": 7867,
7 + "document_hash": "23053422d45849043d9eb3706daf105a581cd66b69e96220ab1285c7c92b2058",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00237_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00237_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.21, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 78.67, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 39.59, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 23.76, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 26.17, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 31.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 − "char_len": 8024,
7 − "document_hash": "455d86e6a8f1e9be0eaebb2e2ab279c0439c6768273a410e7994bc6a76dfbecf",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 37, \"unit_price\": 64.21, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 5}, {\"region\": \"Ouest\", \"product\": \"zeta\", \"units\": 13, \"unit_price\": 78.67, \"discount_pct\": 0, \"returned\": true, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 19, \"unit_price\": 39.59, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"epsilon\", \"units\": 18, \"unit_price\": 23.76, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Est\", \"product\": \"delta\", \"units\": 20, \"unit_price\": 26.17, \"discount_pct\": 25, \"returned\": true, \"shipped_days\": 14}, {\"region\": \"Nord\", \"product\": \"beta\", \"units\": 19, \"unit_price\": 31.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 3}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.",
6 + "char_len": 7842,
7 + "document_hash": "a8320d4e9d153e4398af56ca92c73f39bb1c1b9e97609e9ec096c1157cebe872",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00238_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00238_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 46.52, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 14.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 18.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 15.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 − "char_len": 7855,
7 − "document_hash": "ffdd32b5c64c4cab201265cf785b962e0e60f2f65cf46cfd2674256d0a97fe14",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Nord\", \"product\": \"gamma\", \"units\": 12, \"unit_price\": 46.52, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Ouest\", \"product\": \"epsilon\", \"units\": 31, \"unit_price\": 14.59, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 39, \"unit_price\": 18.43, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"epsilon\", \"units\": 8, \"unit_price\": 15.63, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 1}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 25, \"unit_price\": 74.62, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Ouest\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.36, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 11}]\n\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 0)\n\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.",
6 + "char_len": 7676,
7 + "document_hash": "068a11a09e40cb07626e21d7abb095fb9c4779acc1a0db602ca10678ae79e3e0",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00239_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00239_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 51.01, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 47.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 14.87, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 68.34, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 15.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 10.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7905,
7 − "document_hash": "1215eafbd18b1d396d2779e2c0227bfcff05e59f1f06d602fc8284f18da6bbc8",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 27, \"unit_price\": 51.01, \"discount_pct\": 5, \"returned\": true, \"shipped_days\": 10}, {\"region\": \"Nord\", \"product\": \"alpha\", \"units\": 32, \"unit_price\": 47.91, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 2}, {\"region\": \"Ouest\", \"product\": \"alpha\", \"units\": 2, \"unit_price\": 14.87, \"discount_pct\": 10, \"returned\": false, \"shipped_days\": 14}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 37, \"unit_price\": 68.34, \"discount_pct\": 10, \"returned\": true, \"shipped_days\": 5}, {\"region\": \"Est\", \"product\": \"zeta\", \"units\": 18, \"unit_price\": 15.1, \"discount_pct\": 15, \"returned\": false, \"shipped_days\": 10}, {\"region\": \"Centre\", \"product\": \"alpha\", \"units\": 18, \"unit_price\": 10.35, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}]\n\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nDeployment to production requires a reviewer approval and a green pipeline.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7723,
7 + "document_hash": "015c78e1e480fb805976a26912f0308a79a0c4b2122313463b5ff6b5ee996a9b",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/docs/exec/exec_00240_2000.json +3 −3
@@ -2,9 +2,9 @@
2 2 "document_id": "exec_00240_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "text": "ENGINEERING TICKET\n\nBug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 38.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 28.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 23, \"unit_price\": 35.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 75.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.6, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 35, \"unit_price\": 18.22, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct']\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 − "char_len": 7976,
7 − "document_hash": "2598ae3b4041685f7fbf0de4eb4c12a5508a3c374ae89cdf7007d3cf7e6e5924",
5 + "text": "ENGINEERING TICKET\n\nBug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.\n\nSample rows (first 6 of the extract):\n[{\"region\": \"Sud\", \"product\": \"alpha\", \"units\": 17, \"unit_price\": 38.92, \"discount_pct\": 5, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 40, \"unit_price\": 28.81, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 8}, {\"region\": \"Est\", \"product\": \"beta\", \"units\": 23, \"unit_price\": 35.65, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Est\", \"product\": \"alpha\", \"units\": 30, \"unit_price\": 75.57, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 13}, {\"region\": \"Sud\", \"product\": \"delta\", \"units\": 15, \"unit_price\": 22.6, \"discount_pct\": 0, \"returned\": false, \"shipped_days\": 4}, {\"region\": \"Sud\", \"product\": \"gamma\", \"units\": 35, \"unit_price\": 18.22, \"discount_pct\": 25, \"returned\": false, \"shipped_days\": 6}]\n\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\n\nThe existing (buggy) implementation is:\n\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 1)\n\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nDeployment to production requires a reviewer approval and a green pipeline.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nCode style follows the repository's formatter settings; do not reformat unrelated lines.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe ticketing system assigns severity on a four-level scale; this ticket is severity 2.\nHistoric tickets BR-1183 and BR-2047 are unrelated and remain closed.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nDeployment to production requires a reviewer approval and a green pipeline.\nAll monetary values are in the client's reporting currency; no FX conversion is applied here.\nUnit tests live in the tests/ directory and are executed in CI on every push.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.\nThe data engineering team refreshes the sales extract nightly at 02:00 local time.\nThe data dictionary defines shipped_days as the number of calendar days from order to shipment.\nDeployment to production requires a reviewer approval and a green pipeline.",
6 + "char_len": 7797,
7 + "document_hash": "4c3927b8378a2b949e85371101d959a0adebcbe41a2d39f239aae883de382e19",
8 8 "questions": [
9 9 {
10 10 "question_id": "q00_exec",
modified data/generated/keys/exec/exec_00001_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00001_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "ddfe7fcc3baca3e97144b66188f285a0e8a83018d1ca41b7f37cb80dede62dcc",
5 + "document_hash": "72a702cc7076fb313f97772498675d83532bf6ce59f1d3cbf3bb86fa387dc0e6",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7019: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 751
30 + "char_end": 624
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 7
125 125 }
126 126 ],
127 − "expected": 3629.0
127 + "expected": 3711.0
128 128 },
129 129 {
130 130 "input": [
@@ -269,7 +269,7 @@
269 269 "shipped_days": 7
270 270 }
271 271 ],
272 − "expected": 4601.0
272 + "expected": 4683.0
273 273 },
274 274 {
275 275 "input": [
@@ -400,7 +400,7 @@
400 400 "shipped_days": 5
401 401 }
402 402 ],
403 − "expected": 4660.0
403 + "expected": 4778.0
404 404 },
405 405 {
406 406 "input": [
@@ -459,7 +459,7 @@
459 459 "shipped_days": 2
460 460 }
461 461 ],
462 − "expected": 2896.0
462 + "expected": 2962.0
463 463 }
464 464 ]
465 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00002_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00002_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "7918e3cdcab2a8bfae77558b7b523f4cea70cdc52435393d98d0c7b36de29e60",
5 + "document_hash": "7ec85eb17fda1c57c2a77a31dc79bf35b3c62974f6f1905e409880cbe869b6f5",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6302: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 3
125 125 }
126 126 ],
127 − "expected": 2969.0
127 + "expected": 3084.0
128 128 },
129 129 {
130 130 "input": [
@@ -165,7 +165,7 @@
165 165 "shipped_days": 11
166 166 }
167 167 ],
168 − "expected": 752.0
168 + "expected": 794.0
169 169 },
170 170 {
171 171 "input": [
@@ -233,7 +233,7 @@
233 233 "shipped_days": 11
234 234 }
235 235 ],
236 − "expected": 2144.0
236 + "expected": 2258.0
237 237 },
238 238 {
239 239 "input": [
@@ -391,7 +391,7 @@
391 391 "shipped_days": 5
392 392 }
393 393 ],
394 − "expected": 8977.0
394 + "expected": 9175.0
395 395 },
396 396 {
397 397 "input": [
@@ -531,7 +531,7 @@
531 531 "shipped_days": 5
532 532 }
533 533 ],
534 − "expected": 7566.0
534 + "expected": 7764.0
535 535 }
536 536 ]
537 537 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00003_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00003_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "8ab845867bddfdcb5cf63b1319a6f29794cc9dddea45cd92bca3bc94fe023da7",
5 + "document_hash": "f7d729b32469d142625681408a1dd1aa3d1d99a7c07adb645c9f40d7550e8695",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6542: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 115 "shipped_days": 13
116 116 }
117 117 ],
118 − "expected": 3222.6
118 + "expected": 3349.8
119 119 },
120 120 {
121 121 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 13
229 229 }
230 230 ],
231 − "expected": 6782.7
231 + "expected": 7004.7
232 232 },
233 233 {
234 234 "input": [
@@ -269,7 +269,7 @@
269 269 "shipped_days": 13
270 270 }
271 271 ],
272 − "expected": 1236.8
272 + "expected": 1248.8
273 273 },
274 274 {
275 275 "input": [
@@ -391,7 +391,7 @@
391 391 "shipped_days": 13
392 392 }
393 393 ],
394 − "expected": 7525.7
394 + "expected": 7800.7
395 395 },
396 396 {
397 397 "input": [
@@ -486,7 +486,7 @@
486 486 "shipped_days": 14
487 487 }
488 488 ],
489 − "expected": 6066.7
489 + "expected": 6309.5
490 490 }
491 491 ]
492 492 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00004_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00004_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "08c5cdfaa7a0742a094ee4726d97a32a89395c354270406bbcf78eff4fdbe8b2",
5 + "document_hash": "aeca575de77be2cf25f1c9f8e053485ad918ef4b3468680a4ef6bfa3b8cb781b",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4473: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 761
30 + "char_end": 634
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 97 "shipped_days": 9
98 98 }
99 99 ],
100 − "expected": 5428.38
100 + "expected": 5736.55
101 101 },
102 102 {
103 103 "input": [
@@ -147,7 +147,7 @@
147 147 "shipped_days": 2
148 148 }
149 149 ],
150 − "expected": 6465.92
150 + "expected": 6671.26
151 151 },
152 152 {
153 153 "input": [
@@ -251,7 +251,7 @@
251 251 "shipped_days": 9
252 252 }
253 253 ],
254 − "expected": 10962.43
254 + "expected": 11280.56
255 255 },
256 256 {
257 257 "input": [
@@ -310,7 +310,7 @@
310 310 "shipped_days": 2
311 311 }
312 312 ],
313 − "expected": 5425.82
313 + "expected": 5570.86
314 314 },
315 315 {
316 316 "input": [
@@ -441,7 +441,7 @@
441 441 "shipped_days": 12
442 442 }
443 443 ],
444 − "expected": 9991.84
444 + "expected": 10440.76
445 445 }
446 446 ]
447 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00005_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00005_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "f885d791d02ed1adb010c9c27804b5c38facf8fd2a09b9b1b6f9cacb7b0acc81",
5 + "document_hash": "c5e26772985f1897f27e3729a56bdf95e08ba4718b928a1480572409c422c83f",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1004: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 751
30 + "char_end": 624
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -196,7 +196,7 @@
196 196 "shipped_days": 5
197 197 }
198 198 ],
199 − "expected": 10831.0
199 + "expected": 11286.0
200 200 },
201 201 {
202 202 "input": [
@@ -354,7 +354,7 @@
354 354 "shipped_days": 11
355 355 }
356 356 ],
357 − "expected": 9939.0
357 + "expected": 10311.0
358 358 },
359 359 {
360 360 "input": [
@@ -413,7 +413,7 @@
413 413 "shipped_days": 8
414 414 }
415 415 ],
416 − "expected": 4721.0
416 + "expected": 4918.0
417 417 },
418 418 {
419 419 "input": [
@@ -598,7 +598,7 @@
598 598 "shipped_days": 10
599 599 }
600 600 ],
601 − "expected": 15203.0
601 + "expected": 15877.0
602 602 },
603 603 {
604 604 "input": [
@@ -675,7 +675,7 @@
675 675 "shipped_days": 5
676 676 }
677 677 ],
678 − "expected": 4292.0
678 + "expected": 4542.0
679 679 }
680 680 ]
681 681 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00006_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00006_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "2a1bb6d86c77861b166e281bb1deac9a658ed4383a7ade93e9698e33861de0e5",
5 + "document_hash": "283461ef1dfaf2225620026d081a07116ec5fa2173ffa118704c0cf1d53a2eac",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2483: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 758
30 + "char_end": 631
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 169 "shipped_days": 4
170 170 }
171 171 ],
172 − "expected": 13971.0
172 + "expected": 14483.0
173 173 },
174 174 {
175 175 "input": [
@@ -291,7 +291,7 @@
291 291 "shipped_days": 3
292 292 }
293 293 ],
294 − "expected": 13520.0
294 + "expected": 14032.0
295 295 },
296 296 {
297 297 "input": [
@@ -422,7 +422,7 @@
422 422 "shipped_days": 10
423 423 }
424 424 ],
425 − "expected": 11047.0
425 + "expected": 11404.0
426 426 },
427 427 {
428 428 "input": [
@@ -526,7 +526,7 @@
526 526 "shipped_days": 13
527 527 }
528 528 ],
529 − "expected": 10526.0
529 + "expected": 11032.0
530 530 },
531 531 {
532 532 "input": [
@@ -621,7 +621,7 @@
621 621 "shipped_days": 4
622 622 }
623 623 ],
624 − "expected": 9516.0
624 + "expected": 9886.0
625 625 }
626 626 ]
627 627 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00007_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00007_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "fb0150ce17a4b0ade1601cf6082b82e83f67db83a57c177064777d028b33d97c",
5 + "document_hash": "961b89accd7b0332a165c291bb3c527fd00bea183807cff965326b5f758f1b7f",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6853: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 3 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 759
30 + "char_end": 632
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 160 "shipped_days": 8
161 161 }
162 162 ],
163 − "expected": 5439.03
163 + "expected": 5689.66
164 164 },
165 165 {
166 166 "input": [
@@ -201,7 +201,7 @@
201 201 "shipped_days": 11
202 202 }
203 203 ],
204 − "expected": 2512.96
204 + "expected": 2661.57
205 205 },
206 206 {
207 207 "input": [
@@ -260,7 +260,7 @@
260 260 "shipped_days": 8
261 261 }
262 262 ],
263 − "expected": 1312.49
263 + "expected": 1389.42
264 264 },
265 265 {
266 266 "input": [
@@ -391,7 +391,7 @@
391 391 "shipped_days": 1
392 392 }
393 393 ],
394 − "expected": 6947.25
394 + "expected": 7280.02
395 395 },
396 396 {
397 397 "input": [
@@ -441,7 +441,7 @@
441 441 "shipped_days": 8
442 442 }
443 443 ],
444 − "expected": 3350.53
444 + "expected": 3547.57
445 445 }
446 446 ]
447 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00008_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00008_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "04f896fa681f3ae7cbfc7efa30fd96c43be9d9b703dab76457ee8043483b3612",
5 + "document_hash": "4ad9a1fbe7db8f523841d367742505a5c92581010a15fdabb2bb6b6d77afbaa4",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9591: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 5
134 134 }
135 135 ],
136 − "expected": 6146.0
136 + "expected": 6369.0
137 137 },
138 138 {
139 139 "input": [
@@ -264,7 +264,7 @@
264 264 "shipped_days": 6
265 265 }
266 266 ],
267 − "expected": 7379.0
267 + "expected": 7689.0
268 268 },
269 269 {
270 270 "input": [
@@ -323,7 +323,7 @@
323 323 "shipped_days": 14
324 324 }
325 325 ],
326 − "expected": 4608.0
326 + "expected": 4853.0
327 327 },
328 328 {
329 329 "input": [
@@ -382,7 +382,7 @@
382 382 "shipped_days": 1
383 383 }
384 384 ],
385 − "expected": 3065.0
385 + "expected": 3218.0
386 386 },
387 387 {
388 388 "input": [
@@ -513,7 +513,7 @@
513 513 "shipped_days": 2
514 514 }
515 515 ],
516 − "expected": 7379.0
516 + "expected": 7689.0
517 517 }
518 518 ]
519 519 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00009_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00009_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "db44cbab6b33239dddaa205976960167be792c1b1025bd39a4c4d6cf0f6ab7d7",
5 + "document_hash": "7c7609e3e3780713446d7ec4cbe48dba3ad69a1bf6d96393eb7e34fe13581d81",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4251: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 1
134 134 }
135 135 ],
136 − "expected": 6224.0
136 + "expected": 6375.0
137 137 },
138 138 {
139 139 "input": [
@@ -273,7 +273,7 @@
273 273 "shipped_days": 1
274 274 }
275 275 ],
276 − "expected": 8620.0
276 + "expected": 8850.0
277 277 },
278 278 {
279 279 "input": [
@@ -368,7 +368,7 @@
368 368 "shipped_days": 14
369 369 }
370 370 ],
371 − "expected": 3550.0
371 + "expected": 3637.0
372 372 },
373 373 {
374 374 "input": [
@@ -508,7 +508,7 @@
508 508 "shipped_days": 8
509 509 }
510 510 ],
511 − "expected": 7854.0
511 + "expected": 8104.0
512 512 },
513 513 {
514 514 "input": [
@@ -585,7 +585,7 @@
585 585 "shipped_days": 8
586 586 }
587 587 ],
588 − "expected": 5171.0
588 + "expected": 5396.0
589 589 }
590 590 ]
591 591 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00010_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00010_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "232fe52471a905a3ae14b591781c17dfa57a2d115258bfc1f8a1fce231b705ad",
5 + "document_hash": "2951e556f1045edd4bb9926de1dac81793260ddcd823e8c6abe38566b4cbcc07",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7892: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 751
30 + "char_end": 624
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 11
125 125 }
126 126 ],
127 − "expected": 2617.0
127 + "expected": 2715.0
128 128 },
129 129 {
130 130 "input": [
@@ -183,7 +183,7 @@
183 183 "shipped_days": 6
184 184 }
185 185 ],
186 − "expected": 1214.0
186 + "expected": 1237.0
187 187 },
188 188 {
189 189 "input": [
@@ -323,7 +323,7 @@
323 323 "shipped_days": 7
324 324 }
325 325 ],
326 − "expected": 3768.0
326 + "expected": 3907.0
327 327 },
328 328 {
329 329 "input": [
@@ -382,7 +382,7 @@
382 382 "shipped_days": 10
383 383 }
384 384 ],
385 − "expected": 694.0
385 + "expected": 724.0
386 386 },
387 387 {
388 388 "input": [
@@ -567,7 +567,7 @@
567 567 "shipped_days": 11
568 568 }
569 569 ],
570 − "expected": 5560.0
570 + "expected": 5735.0
571 571 }
572 572 ]
573 573 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00011_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00011_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "f046c8292307adf97cae067d41f5a7bcce9d2a48bc8dd143e0f11e89a1c3a4c2",
5 + "document_hash": "6b5b870bbcdf744c667890cf6185717b10f87e45ad7a6ccada501987d53f5e6d",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1386: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 23 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 753
30 + "char_end": 626
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 13
89 89 }
90 90 ],
91 − "expected": 3443.0
91 + "expected": 3644.0
92 92 },
93 93 {
94 94 "input": [
@@ -192,7 +192,7 @@
192 192 "shipped_days": 13
193 193 }
194 194 ],
195 − "expected": 4208.0
195 + "expected": 4474.0
196 196 },
197 197 {
198 198 "input": [
@@ -287,7 +287,7 @@
287 287 "shipped_days": 14
288 288 }
289 289 ],
290 − "expected": 3787.0
290 + "expected": 4024.0
291 291 },
292 292 {
293 293 "input": [
@@ -418,7 +418,7 @@
418 418 "shipped_days": 7
419 419 }
420 420 ],
421 − "expected": 5822.0
421 + "expected": 6177.0
422 422 },
423 423 {
424 424 "input": [
@@ -540,7 +540,7 @@
540 540 "shipped_days": 14
541 541 }
542 542 ],
543 − "expected": 5822.0
543 + "expected": 6177.0
544 544 }
545 545 ]
546 546 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00012_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00012_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "2befcba5bfd5b509a9a6ec5d9334e2e384a05b6733ac02d80222302d3ca6e6ef",
5 + "document_hash": "8991dd153dfd83b4a1aea0eb4340aabe4370ac72ef16427f5f67894ac175edde",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-3390: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -196,7 +196,7 @@
196 196 "shipped_days": 13
197 197 }
198 198 ],
199 − "expected": 6955.0
199 + "expected": 7281.0
200 200 },
201 201 {
202 202 "input": [
@@ -237,7 +237,7 @@
237 237 "shipped_days": 10
238 238 }
239 239 ],
240 − "expected": 2047.0
240 + "expected": 2140.0
241 241 },
242 242 {
243 243 "input": [
@@ -323,7 +323,7 @@
323 323 "shipped_days": 2
324 324 }
325 325 ],
326 − "expected": 4220.0
326 + "expected": 4380.0
327 327 },
328 328 {
329 329 "input": [
@@ -490,7 +490,7 @@
490 490 "shipped_days": 8
491 491 }
492 492 ],
493 − "expected": 7371.0
493 + "expected": 7658.0
494 494 },
495 495 {
496 496 "input": [
@@ -603,7 +603,7 @@
603 603 "shipped_days": 2
604 604 }
605 605 ],
606 − "expected": 3079.0
606 + "expected": 3117.0
607 607 }
608 608 ]
609 609 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00013_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00013_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "d006207e385cff8703be29071faf5845de84d05d8a677e40dda4983739115d29",
5 + "document_hash": "ee93ee22294a4463cb6cf9ca3d5ab7ea7fd2a991b2b55448f089c2290e51a174",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8332: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 753
30 + "char_end": 626
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -151,7 +151,7 @@
151 151 "shipped_days": 2
152 152 }
153 153 ],
154 − "expected": 5458.0
154 + "expected": 5773.0
155 155 },
156 156 {
157 157 "input": [
@@ -219,7 +219,7 @@
219 219 "shipped_days": 2
220 220 }
221 221 ],
222 − "expected": 1842.0
222 + "expected": 1944.0
223 223 },
224 224 {
225 225 "input": [
@@ -296,7 +296,7 @@
296 296 "shipped_days": 8
297 297 }
298 298 ],
299 − "expected": 4882.0
299 + "expected": 5107.0
300 300 },
301 301 {
302 302 "input": [
@@ -445,7 +445,7 @@
445 445 "shipped_days": 14
446 446 }
447 447 ],
448 − "expected": 7221.0
448 + "expected": 7577.0
449 449 },
450 450 {
451 451 "input": [
@@ -504,7 +504,7 @@
504 504 "shipped_days": 9
505 505 }
506 506 ],
507 − "expected": 2757.0
507 + "expected": 2864.0
508 508 }
509 509 ]
510 510 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00014_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00014_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "e6ac9234df3fc7dcec86770e041bf75b576d1c65cf6fe32717891429f5461066",
5 + "document_hash": "b25f9d4fb9c997b0cb6e19983b48490097f6592b8d17e66f305f5a1b3e8a8443",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7580: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 160 "shipped_days": 5
161 161 }
162 162 ],
163 − "expected": 8460.0
163 + "expected": 8930.0
164 164 },
165 165 {
166 166 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 5
229 229 }
230 230 ],
231 − "expected": 2370.0
231 + "expected": 2505.0
232 232 },
233 233 {
234 234 "input": [
@@ -305,7 +305,7 @@
305 305 "shipped_days": 4
306 306 }
307 307 ],
308 − "expected": 3607.0
308 + "expected": 3812.0
309 309 },
310 310 {
311 311 "input": [
@@ -454,7 +454,7 @@
454 454 "shipped_days": 13
455 455 }
456 456 ],
457 − "expected": 11029.0
457 + "expected": 11639.0
458 458 },
459 459 {
460 460 "input": [
@@ -558,7 +558,7 @@
558 558 "shipped_days": 4
559 559 }
560 560 ],
561 − "expected": 9013.0
561 + "expected": 9499.0
562 562 }
563 563 ]
564 564 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00015_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00015_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "7bd733e903db470812271aa82f1cd9c7111ace0dcd257ace35b07e660d045be7",
5 + "document_hash": "7334bc5b260e11b2b0cdf9db05839cac2f397ae0e44db4e8bba7559dc94036bb",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 5, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7400: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 106 "shipped_days": 10
107 107 }
108 108 ],
109 − "expected": 2391.96
109 + "expected": 2525.61
110 110 },
111 111 {
112 112 "input": [
@@ -219,7 +219,7 @@
219 219 "shipped_days": 10
220 220 }
221 221 ],
222 − "expected": 4322.59
222 + "expected": 4529.08
223 223 },
224 224 {
225 225 "input": [
@@ -332,7 +332,7 @@
332 332 "shipped_days": 11
333 333 }
334 334 ],
335 − "expected": 2997.99
335 + "expected": 3122.6
336 336 },
337 337 {
338 338 "input": [
@@ -454,7 +454,7 @@
454 454 "shipped_days": 1
455 455 }
456 456 ],
457 − "expected": 3013.36
457 + "expected": 3138.94
458 458 },
459 459 {
460 460 "input": [
@@ -585,7 +585,7 @@
585 585 "shipped_days": 10
586 586 }
587 587 ],
588 − "expected": 4571.24
588 + "expected": 4788.45
589 589 }
590 590 ]
591 591 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00016_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00016_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "b232fee620440e55facf033b892741387c97661b269dc2371313fb0f5934ff69",
5 + "document_hash": "6a5268852caa8dc3015fd42e678e912f463ba061ac417b09de9ac6ed45e85d7f",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 6, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8263: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 751
30 + "char_end": 624
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -187,7 +187,7 @@
187 187 "shipped_days": 12
188 188 }
189 189 ],
190 − "expected": 7227.8
190 + "expected": 7417.1
191 191 },
192 192 {
193 193 "input": [
@@ -273,7 +273,7 @@
273 273 "shipped_days": 2
274 274 }
275 275 ],
276 − "expected": 4555.1
276 + "expected": 4666.4
277 277 },
278 278 {
279 279 "input": [
@@ -364,7 +364,7 @@
364 364 "shipped_days": 6
365 365 }
366 366 ],
367 − "expected": 934.1
367 + "expected": 986.0
368 368 },
369 369 {
370 370 "input": [
@@ -405,7 +405,7 @@
405 405 "shipped_days": 5
406 406 }
407 407 ],
408 − "expected": 2449.0
408 + "expected": 2494.6
409 409 }
410 410 ]
411 411 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00017_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00017_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "bce532bd3702b32e6eb3620ef3ea7d860a695f22bb90b8e995836a7937c77f52",
5 + "document_hash": "f2d06422bd98b400b7401ce4c5ae2939aeb43f51482e84cba0c0e3c0845c71a5",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-5234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 13
134 134 }
135 135 ],
136 − "expected": 9108.6
136 + "expected": 9456.8
137 137 },
138 138 {
139 139 "input": [
@@ -237,7 +237,7 @@
237 237 "shipped_days": 6
238 238 }
239 239 ],
240 − "expected": 9400.8
240 + "expected": 9760.9
241 241 },
242 242 {
243 243 "input": [
@@ -413,7 +413,7 @@
413 413 "shipped_days": 1
414 414 }
415 415 ],
416 − "expected": 15622.3
416 + "expected": 16236.8
417 417 },
418 418 {
419 419 "input": [
@@ -589,7 +589,7 @@
589 589 "shipped_days": 1
590 590 }
591 591 ],
592 − "expected": 14227.3
592 + "expected": 14862.0
593 593 },
594 594 {
595 595 "input": [
@@ -756,7 +756,7 @@
756 756 "shipped_days": 8
757 757 }
758 758 ],
759 − "expected": 15622.3
759 + "expected": 16236.8
760 760 }
761 761 ]
762 762 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00018_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00018_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "6f0951d7af338593059efbe431f0b7ce5bf9ae652e3dfb9eedafa7e192b80c01",
5 + "document_hash": "f219110fda4e5d20fabbe592248da3e78f61f411dbc16b9412d9b8d692eda870",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-3864: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 160 "shipped_days": 5
161 161 }
162 162 ],
163 − "expected": 9488.91
163 + "expected": 9778.64
164 164 },
165 165 {
166 166 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 10
229 229 }
230 230 ],
231 − "expected": 4726.7
231 + "expected": 4840.02
232 232 },
233 233 {
234 234 "input": [
@@ -395,7 +395,7 @@
395 395 "shipped_days": 7
396 396 }
397 397 ],
398 − "expected": 12483.47
398 + "expected": 12743.47
399 399 },
400 400 {
401 401 "input": [
@@ -580,7 +580,7 @@
580 580 "shipped_days": 12
581 581 }
582 582 ],
583 − "expected": 14020.45
583 + "expected": 14355.94
584 584 },
585 585 {
586 586 "input": [
@@ -648,7 +648,7 @@
648 648 "shipped_days": 4
649 649 }
650 650 ],
651 − "expected": 5979.52
651 + "expected": 6238.62
652 652 }
653 653 ]
654 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00019_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00019_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "733853d52a559dfc67d0cdab02de9e21c2aa09e0b25702d2cadfb38c20ab2159",
5 + "document_hash": "232819b55ebc60215845a14aaf01c04a94dc7902933c03a47546ec41ee16502f",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4945: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -151,7 +151,7 @@
151 151 "shipped_days": 5
152 152 }
153 153 ],
154 − "expected": 6814.82
154 + "expected": 7072.24
155 155 },
156 156 {
157 157 "input": [
@@ -318,7 +318,7 @@
318 318 "shipped_days": 12
319 319 }
320 320 ],
321 − "expected": 11999.36
321 + "expected": 12263.98
322 322 },
323 323 {
324 324 "input": [
@@ -404,7 +404,7 @@
404 404 "shipped_days": 6
405 405 }
406 406 ],
407 − "expected": 4050.38
407 + "expected": 4142.87
408 408 },
409 409 {
410 410 "input": [
@@ -544,7 +544,7 @@
544 544 "shipped_days": 1
545 545 }
546 546 ],
547 − "expected": 12701.97
547 + "expected": 13019.21
548 548 },
549 549 {
550 550 "input": [
@@ -630,7 +630,7 @@
630 630 "shipped_days": 7
631 631 }
632 632 ],
633 − "expected": 5014.11
633 + "expected": 5141.41
634 634 }
635 635 ]
636 636 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00020_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00020_2000",
3 3 "family": "exec",
4 4 "split": "train",
5 − "document_hash": "1a7bfffc3347254640c8b76b9b5f0ff368a758543995fa06006e28ea43d8eb71",
5 + "document_hash": "862136f6ccb7f77097086ceddd84945e55b7df17401a63b0ce306740969966e5",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8600: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -223,7 +223,7 @@
223 223 "shipped_days": 5
224 224 }
225 225 ],
226 − "expected": 8844.77
226 + "expected": 9290.06
227 227 },
228 228 {
229 229 "input": [
@@ -435,7 +435,7 @@
435 435 "shipped_days": 13
436 436 }
437 437 ],
438 − "expected": 9166.7
438 + "expected": 9612.0
439 439 },
440 440 {
441 441 "input": [
@@ -530,7 +530,7 @@
530 530 "shipped_days": 4
531 531 }
532 532 ],
533 − "expected": 3537.06
533 + "expected": 3697.66
534 534 },
535 535 {
536 536 "input": [
@@ -715,7 +715,7 @@
715 715 "shipped_days": 13
716 716 }
717 717 ],
718 − "expected": 9068.02
718 + "expected": 9513.31
719 719 },
720 720 {
721 721 "input": [
@@ -891,7 +891,7 @@
891 891 "shipped_days": 9
892 892 }
893 893 ],
894 − "expected": 8860.27
894 + "expected": 9289.44
895 895 }
896 896 ]
897 897 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00101_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00101_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9",
5 + "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 2, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1234: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -223,7 +223,7 @@
223 223 "shipped_days": 1
224 224 }
225 225 ],
226 − "expected": 10226.1
226 + "expected": 10668.21
227 227 },
228 228 {
229 229 "input": [
@@ -381,7 +381,7 @@
381 381 "shipped_days": 6
382 382 }
383 383 ],
384 − "expected": 8617.57
384 + "expected": 9061.05
385 385 },
386 386 {
387 387 "input": [
@@ -458,7 +458,7 @@
458 458 "shipped_days": 2
459 459 }
460 460 ],
461 − "expected": 5081.68
461 + "expected": 5250.95
462 462 },
463 463 {
464 464 "input": [
@@ -526,7 +526,7 @@
526 526 "shipped_days": 1
527 527 }
528 528 ],
529 − "expected": 3365.1
529 + "expected": 3549.55
530 530 },
531 531 {
532 532 "input": [
@@ -684,7 +684,7 @@
684 684 "shipped_days": 8
685 685 }
686 686 ],
687 − "expected": 5902.78
687 + "expected": 6132.87
688 688 }
689 689 ]
690 690 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00102_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00102_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "5535ac3109616f7d46dff5417535ca232754aee2c0759e072d552e465f004d46",
5 + "document_hash": "61b6c7bb86261a26bf000996edd5f9326f7f77ca1c37085fd999cddf1bff2396",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4681: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -70,7 +70,7 @@
70 70 "shipped_days": 12
71 71 }
72 72 ],
73 − "expected": 2436.53
73 + "expected": 2584.83
74 74 },
75 75 {
76 76 "input": [
@@ -192,7 +192,7 @@
192 192 "shipped_days": 7
193 193 }
194 194 ],
195 − "expected": 7499.55
195 + "expected": 7698.08
196 196 },
197 197 {
198 198 "input": [
@@ -386,7 +386,7 @@
386 386 "shipped_days": 12
387 387 }
388 388 ],
389 − "expected": 9927.27
389 + "expected": 10303.84
390 390 },
391 391 {
392 392 "input": [
@@ -436,7 +436,7 @@
436 436 "shipped_days": 9
437 437 }
438 438 ],
439 − "expected": 547.82
439 + "expected": 579.97
440 440 },
441 441 {
442 442 "input": [
@@ -495,7 +495,7 @@
495 495 "shipped_days": 3
496 496 }
497 497 ],
498 − "expected": 3172.53
498 + "expected": 3203.35
499 499 }
500 500 ]
501 501 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00103_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00103_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "a7875d1e95742c6246c4ec3862b9797a4269a2b607e47869f5014d7dc29139e0",
5 + "document_hash": "596ef1cd6d37fdac729520a738ed129e4ffaf619d372ad869d8188a0a5cac1ef",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 2, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1178: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 10
134 134 }
135 135 ],
136 − "expected": 6268.0
136 + "expected": 6449.0
137 137 },
138 138 {
139 139 "input": [
@@ -282,7 +282,7 @@
282 282 "shipped_days": 8
283 283 }
284 284 ],
285 − "expected": 8227.0
285 + "expected": 8477.0
286 286 },
287 287 {
288 288 "input": [
@@ -350,7 +350,7 @@
350 350 "shipped_days": 8
351 351 }
352 352 ],
353 − "expected": 5279.0
353 + "expected": 5424.0
354 354 },
355 355 {
356 356 "input": [
@@ -459,7 +459,7 @@
459 459 "shipped_days": 4
460 460 }
461 461 ],
462 − "expected": 5088.0
462 + "expected": 5269.0
463 463 }
464 464 ]
465 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00104_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00104_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "bce3091895fc315bf981ac1272a7da1f00b0f64819f1ecfa2a6dd8a3d23f2207",
5 + "document_hash": "6b4aadaf1b71672d6d8aed5a412c7aa490e3e0eb1c92e5df6db1cc9554f83483",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6532: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 9
89 89 }
90 90 ],
91 − "expected": 2111.0
91 + "expected": 2225.0
92 92 },
93 93 {
94 94 "input": [
@@ -138,7 +138,7 @@
138 138 "shipped_days": 8
139 139 }
140 140 ],
141 − "expected": 2304.0
141 + "expected": 2439.0
142 142 },
143 143 {
144 144 "input": [
@@ -215,7 +215,7 @@
215 215 "shipped_days": 10
216 216 }
217 217 ],
218 − "expected": 6214.0
218 + "expected": 6444.0
219 219 },
220 220 {
221 221 "input": [
@@ -373,7 +373,7 @@
373 373 "shipped_days": 12
374 374 }
375 375 ],
376 − "expected": 8777.0
376 + "expected": 9137.0
377 377 },
378 378 {
379 379 "input": [
@@ -441,7 +441,7 @@
441 441 "shipped_days": 3
442 442 }
443 443 ],
444 − "expected": 3400.0
444 + "expected": 3456.0
445 445 }
446 446 ]
447 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00105_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00105_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "fe4b8857bf1774be1123bec85a5743da5c3551d53125ddf128efde401f3990c2",
5 + "document_hash": "a59face90fe592d3d8a9b375aafefb775631dd798a60a424f981cceae1ec2996",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2817: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 1
134 134 }
135 135 ],
136 − "expected": 10771.09
136 + "expected": 10902.26
137 137 },
138 138 {
139 139 "input": [
@@ -237,7 +237,7 @@
237 237 "shipped_days": 3
238 238 }
239 239 ],
240 − "expected": 10110.37
240 + "expected": 10248.58
241 241 },
242 242 {
243 243 "input": [
@@ -359,7 +359,7 @@
359 359 "shipped_days": 11
360 360 }
361 361 ],
362 − "expected": 11426.01
362 + "expected": 11578.59
363 363 },
364 364 {
365 365 "input": [
@@ -445,7 +445,7 @@
445 445 "shipped_days": 10
446 446 }
447 447 ],
448 − "expected": 7450.92
448 + "expected": 7589.13
449 449 },
450 450 {
451 451 "input": [
@@ -513,7 +513,7 @@
513 513 "shipped_days": 8
514 514 }
515 515 ],
516 − "expected": 7375.01
516 + "expected": 7509.36
517 517 }
518 518 ]
519 519 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00106_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00106_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "c3652bfa1632f0cc62d6512131ef519100143881225c8b4ee572c1876e7c6c7f",
5 + "document_hash": "a78f55af374fd04eb589b193a9eca2f84cf04bcdb3fa60aaeb29fbe358c79464",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8363: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 758
30 + "char_end": 631
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 79 "shipped_days": 8
80 80 }
81 81 ],
82 − "expected": 5525.0
82 + "expected": 5732.0
83 83 },
84 84 {
85 85 "input": [
@@ -174,7 +174,7 @@
174 174 "shipped_days": 2
175 175 }
176 176 ],
177 − "expected": 6501.0
177 + "expected": 6608.0
178 178 },
179 179 {
180 180 "input": [
@@ -287,7 +287,7 @@
287 287 "shipped_days": 2
288 288 }
289 289 ],
290 − "expected": 10681.0
290 + "expected": 11051.0
291 291 },
292 292 {
293 293 "input": [
@@ -337,7 +337,7 @@
337 337 "shipped_days": 1
338 338 }
339 339 ],
340 − "expected": 3956.0
340 + "expected": 4119.0
341 341 },
342 342 {
343 343 "input": [
@@ -468,7 +468,7 @@
468 468 "shipped_days": 14
469 469 }
470 470 ],
471 − "expected": 11106.0
471 + "expected": 11475.0
472 472 }
473 473 ]
474 474 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00107_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00107_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "983dbcacdfae2acf0316690a7fe50cfa2523ae41beffd9de42356f2ebd3d1e79",
5 + "document_hash": "83ea6e4f6c5d3a6eed7a921a930819d760ffb3dac7c22761dc9b4c28084904cc",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7434: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 106 "shipped_days": 14
107 107 }
108 108 ],
109 − "expected": 5613.04
109 + "expected": 5823.85
110 110 },
111 111 {
112 112 "input": [
@@ -210,7 +210,7 @@
210 210 "shipped_days": 3
211 211 }
212 212 ],
213 − "expected": 3646.24
213 + "expected": 3759.3
214 214 },
215 215 {
216 216 "input": [
@@ -260,7 +260,7 @@
260 260 "shipped_days": 13
261 261 }
262 262 ],
263 − "expected": 1237.55
263 + "expected": 1292.08
264 264 },
265 265 {
266 266 "input": [
@@ -373,7 +373,7 @@
373 373 "shipped_days": 3
374 374 }
375 375 ],
376 − "expected": 6615.56
376 + "expected": 6884.9
377 377 },
378 378 {
379 379 "input": [
@@ -459,7 +459,7 @@
459 459 "shipped_days": 4
460 460 }
461 461 ],
462 − "expected": 3551.44
462 + "expected": 3657.72
463 463 }
464 464 ]
465 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00108_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00108_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "7ebb05d3714e237d074dfc9a69c5b83e99932f9041b5d4fe15c8fc7224ba7078",
5 + "document_hash": "b9ccaaf72598ea6e12a10942f2697dd01ca891115b652043a2f6ce4cb68570cc",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-5898: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 169 "shipped_days": 8
170 170 }
171 171 ],
172 − "expected": 4576.0
172 + "expected": 4658.0
173 173 },
174 174 {
175 175 "input": [
@@ -345,7 +345,7 @@
345 345 "shipped_days": 1
346 346 }
347 347 ],
348 − "expected": 8240.0
348 + "expected": 8534.0
349 349 },
350 350 {
351 351 "input": [
@@ -476,7 +476,7 @@
476 476 "shipped_days": 6
477 477 }
478 478 ],
479 − "expected": 8190.0
479 + "expected": 8483.0
480 480 },
481 481 {
482 482 "input": [
@@ -553,7 +553,7 @@
553 553 "shipped_days": 13
554 554 }
555 555 ],
556 − "expected": 4773.0
556 + "expected": 4985.0
557 557 },
558 558 {
559 559 "input": [
@@ -702,7 +702,7 @@
702 702 "shipped_days": 1
703 703 }
704 704 ],
705 − "expected": 8587.0
705 + "expected": 8880.0
706 706 }
707 707 ]
708 708 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00109_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00109_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "680dab47e67683cb351941eea647197cb2b2ea472ff116582019ace0c543d461",
5 + "document_hash": "d53ada6a135b4827dcacb2ebec142b5000ad21961fb0cd199fc7c609424d4d5a",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1880: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 14 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 79 "shipped_days": 14
80 80 }
81 81 ],
82 − "expected": 3510.0
82 + "expected": 3706.0
83 83 },
84 84 {
85 85 "input": [
@@ -255,7 +255,7 @@
255 255 "shipped_days": 4
256 256 }
257 257 ],
258 − "expected": 11820.0
258 + "expected": 12136.0
259 259 },
260 260 {
261 261 "input": [
@@ -296,7 +296,7 @@
296 296 "shipped_days": 11
297 297 }
298 298 ],
299 − "expected": 1937.0
299 + "expected": 2020.0
300 300 },
301 301 {
302 302 "input": [
@@ -490,7 +490,7 @@
490 490 "shipped_days": 3
491 491 }
492 492 ],
493 − "expected": 12167.0
493 + "expected": 12483.0
494 494 },
495 495 {
496 496 "input": [
@@ -585,7 +585,7 @@
585 585 "shipped_days": 14
586 586 }
587 587 ],
588 − "expected": 6741.0
588 + "expected": 6937.0
589 589 }
590 590 ]
591 591 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00110_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00110_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "f1699da28903736ad58ec0640a277bb2eac030c80c49c22c027c273374df86fa",
5 + "document_hash": "789f1f1a750d514137a988d21a0807cc30c8548da830f2eeef66854fc179fcf5",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 115 "shipped_days": 13
116 116 }
117 117 ],
118 − "expected": 2826.7
118 + "expected": 2987.2
119 119 },
120 120 {
121 121 "input": [
@@ -237,7 +237,7 @@
237 237 "shipped_days": 2
238 238 }
239 239 ],
240 − "expected": 7282.5
240 + "expected": 7731.2
241 241 },
242 242 {
243 243 "input": [
@@ -296,7 +296,7 @@
296 296 "shipped_days": 5
297 297 }
298 298 ],
299 − "expected": 2365.4
299 + "expected": 2478.7
300 300 },
301 301 {
302 302 "input": [
@@ -391,7 +391,7 @@
391 391 "shipped_days": 7
392 392 }
393 393 ],
394 − "expected": 3707.7
394 + "expected": 3903.5
395 395 },
396 396 {
397 397 "input": [
@@ -450,7 +450,7 @@
450 450 "shipped_days": 7
451 451 }
452 452 ],
453 − "expected": 1578.9
453 + "expected": 1670.6
454 454 }
455 455 ]
456 456 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00111_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00111_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "e65e9534b60b15cdaadf465396c6385868f5a0ed4be5ba04c458937c2eff43ae",
5 + "document_hash": "9f3d9d2caa1830b0bf0b983179f4de3251ec9bb8648c710bf43f0fb6c531ef85",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8832: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 750
30 + "char_end": 623
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 79 "shipped_days": 14
80 80 }
81 81 ],
82 − "expected": 4912.9
82 + "expected": 4941.8
83 83 },
84 84 {
85 85 "input": [
@@ -129,7 +129,7 @@
129 129 "shipped_days": 4
130 130 }
131 131 ],
132 − "expected": 4597.7
132 + "expected": 4704.8
133 133 },
134 134 {
135 135 "input": [
@@ -242,7 +242,7 @@
242 242 "shipped_days": 2
243 243 }
244 244 ],
245 − "expected": 7960.9
245 + "expected": 8086.8
246 246 },
247 247 {
248 248 "input": [
@@ -283,7 +283,7 @@
283 283 "shipped_days": 2
284 284 }
285 285 ],
286 − "expected": 4797.6
286 + "expected": 4924.0
287 287 },
288 288 {
289 289 "input": [
@@ -333,7 +333,7 @@
333 333 "shipped_days": 1
334 334 }
335 335 ],
336 − "expected": 2120.4
336 + "expected": 2221.0
337 337 }
338 338 ]
339 339 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00112_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00112_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "4a8db801f5e7f6b2108618514f90c37fbdbb61e902c4441c0d68548a5792da05",
5 + "document_hash": "182cf1cbb5939b7e1ca9170096aa77a9cc3c2c4f8efb3195e8b4ad19b77a3f46",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 5, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1935: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 759
30 + "char_end": 632
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 2
89 89 }
90 90 ],
91 − "expected": 3649.3
91 + "expected": 3759.0
92 92 },
93 93 {
94 94 "input": [
@@ -165,7 +165,7 @@
165 165 "shipped_days": 2
166 166 }
167 167 ],
168 − "expected": 7068.7
168 + "expected": 7383.5
169 169 },
170 170 {
171 171 "input": [
@@ -206,7 +206,7 @@
206 206 "shipped_days": 6
207 207 }
208 208 ],
209 − "expected": 1034.9
209 + "expected": 1091.9
210 210 },
211 211 {
212 212 "input": [
@@ -310,7 +310,7 @@
310 310 "shipped_days": 3
311 311 }
312 312 ],
313 − "expected": 6354.2
313 + "expected": 6637.2
314 314 },
315 315 {
316 316 "input": [
@@ -432,7 +432,7 @@
432 432 "shipped_days": 2
433 433 }
434 434 ],
435 − "expected": 8695.4
435 + "expected": 9104.1
436 436 }
437 437 ]
438 438 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00113_2000.json +6 −6
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00113_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "7535b4d324a3e28b457e40fe41d18d52e75a9ed0271b411285e522c6c8c8200e",
5 + "document_hash": "c89720bd112bca637c8fa3b4a98f7cc41a384be468eee47f5c44607ee6fc46f1",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7037: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 13 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -368,7 +368,7 @@
368 368 "shipped_days": 12
369 369 }
370 370 ],
371 − "expected": 7773.0
371 + "expected": 7875.8
372 372 },
373 373 {
374 374 "input": [
@@ -544,7 +544,7 @@
544 544 "shipped_days": 6
545 545 }
546 546 ],
547 − "expected": 9327.9
547 + "expected": 9430.8
548 548 },
549 549 {
550 550 "input": [
modified data/generated/keys/exec/exec_00114_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00114_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "e1faab8175075b8cf98d2e5e16181db3f5a8f103ed89d35efee7ba4684968699",
5 + "document_hash": "86a377fe8f0559479ca21128972c50d14bd9e19d38d2bf49f763d402259c0aad",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7391: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 7
125 125 }
126 126 ],
127 − "expected": 3625.0
127 + "expected": 3805.0
128 128 },
129 129 {
130 130 "input": [
@@ -183,7 +183,7 @@
183 183 "shipped_days": 6
184 184 }
185 185 ],
186 − "expected": 2378.0
186 + "expected": 2511.0
187 187 },
188 188 {
189 189 "input": [
@@ -224,7 +224,7 @@
224 224 "shipped_days": 1
225 225 }
226 226 ],
227 − "expected": 2182.0
227 + "expected": 2273.0
228 228 },
229 229 {
230 230 "input": [
@@ -265,7 +265,7 @@
265 265 "shipped_days": 7
266 266 }
267 267 ],
268 − "expected": 1604.0
268 + "expected": 1697.0
269 269 },
270 270 {
271 271 "input": [
@@ -324,7 +324,7 @@
324 324 "shipped_days": 1
325 325 }
326 326 ],
327 − "expected": 1570.0
327 + "expected": 1635.0
328 328 }
329 329 ]
330 330 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00115_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00115_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "4d6a9aa137c992493ef42f6758dbb35b873a4a392b00eae7152d6e45b68a3c1b",
5 + "document_hash": "33ee1d20ea5713048e1335d3786ba0b10b3a5998a16200f09823d0c4cf0dcbe1",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 2, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4157: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 97 "shipped_days": 1
98 98 }
99 99 ],
100 − "expected": 6895.0
100 + "expected": 7259.0
101 101 },
102 102 {
103 103 "input": [
@@ -237,7 +237,7 @@
237 237 "shipped_days": 10
238 238 }
239 239 ],
240 − "expected": 8291.0
240 + "expected": 8731.0
241 241 },
242 242 {
243 243 "input": [
@@ -305,7 +305,7 @@
305 305 "shipped_days": 13
306 306 }
307 307 ],
308 − "expected": 1592.0
308 + "expected": 1662.0
309 309 },
310 310 {
311 311 "input": [
@@ -364,7 +364,7 @@
364 364 "shipped_days": 1
365 365 }
366 366 ],
367 − "expected": 1981.0
367 + "expected": 2073.0
368 368 },
369 369 {
370 370 "input": [
@@ -468,7 +468,7 @@
468 468 "shipped_days": 4
469 469 }
470 470 ],
471 − "expected": 2518.0
471 + "expected": 2593.0
472 472 }
473 473 ]
474 474 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00116_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00116_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "2b6100ff4b04865ebf622b9749757f3b32215faa2e32f12d63be71b0c18fc7be",
5 + "document_hash": "379dcf9e9ada65463c8d73e2cbb54ff33c8c46ea3154539a1d6df25bab2a0a0e",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 5, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8712: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 11
125 125 }
126 126 ],
127 − "expected": 3800.17
127 + "expected": 4017.88
128 128 },
129 129 {
130 130 "input": [
@@ -219,7 +219,7 @@
219 219 "shipped_days": 11
220 220 }
221 221 ],
222 − "expected": 3800.17
222 + "expected": 4017.88
223 223 },
224 224 {
225 225 "input": [
@@ -305,7 +305,7 @@
305 305 "shipped_days": 4
306 306 }
307 307 ],
308 − "expected": 2139.86
308 + "expected": 2256.71
309 309 },
310 310 {
311 311 "input": [
@@ -364,7 +364,7 @@
364 364 "shipped_days": 4
365 365 }
366 366 ],
367 − "expected": 2238.1
367 + "expected": 2365.02
368 368 },
369 369 {
370 370 "input": [
@@ -477,7 +477,7 @@
477 477 "shipped_days": 8
478 478 }
479 479 ],
480 − "expected": 3800.17
480 + "expected": 4017.88
481 481 }
482 482 ]
483 483 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00117_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00117_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "6e53d886a35d8e51c98908f1e40e9314353c62a19e3197603509fd2a59c6bbf3",
5 + "document_hash": "34a1cc9614d0c2d2f9766b2968ae3fa3eb2b6cc1a372750650ae7f7359325896",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8902: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 169 "shipped_days": 10
170 170 }
171 171 ],
172 − "expected": 5636.0
172 + "expected": 5781.7
173 173 },
174 174 {
175 175 "input": [
@@ -318,7 +318,7 @@
318 318 "shipped_days": 2
319 319 }
320 320 ],
321 − "expected": 5209.9
321 + "expected": 5379.2
322 322 },
323 323 {
324 324 "input": [
@@ -386,7 +386,7 @@
386 386 "shipped_days": 2
387 387 }
388 388 ],
389 − "expected": 468.5
389 + "expected": 493.1
390 390 },
391 391 {
392 392 "input": [
@@ -553,7 +553,7 @@
553 553 "shipped_days": 3
554 554 }
555 555 ],
556 − "expected": 6038.0
556 + "expected": 6207.3
557 557 },
558 558 {
559 559 "input": [
modified data/generated/keys/exec/exec_00118_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00118_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "5353584056feb451c43bd2199aa87e2f7064bbfb9485b6eb1f6a1387dddac0f6",
5 + "document_hash": "58d2108f993d9baafb67222dc4bd97895639cb6c2410e675e3ed2c4233c8fee4",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8999: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -255,7 +255,7 @@
255 255 "shipped_days": 9
256 256 }
257 257 ],
258 − "expected": 10422.52
258 + "expected": 10515.77
259 259 },
260 260 {
261 261 "input": [
@@ -377,7 +377,7 @@
377 377 "shipped_days": 5
378 378 }
379 379 ],
380 − "expected": 9268.75
380 + "expected": 9362.0
381 381 },
382 382 {
383 383 "input": [
@@ -454,7 +454,7 @@
454 454 "shipped_days": 1
455 455 }
456 456 ],
457 − "expected": 7023.17
457 + "expected": 7059.96
458 458 },
459 459 {
460 460 "input": [
@@ -594,7 +594,7 @@
594 594 "shipped_days": 1
595 595 }
596 596 ],
597 − "expected": 10422.52
597 + "expected": 10515.77
598 598 }
599 599 ]
600 600 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00119_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00119_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "f62c9b5ecd91caa0cf95af49bf6d40dd2bc3885255008434f482e56b7a2a2fae",
5 + "document_hash": "98f121e7a33ff30acfcaf7a0346bbc04cc9f508e092e316490793765a8e0ab09",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6430: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 3
134 134 }
135 135 ],
136 − "expected": 3946.0
136 + "expected": 4007.0
137 137 },
138 138 {
139 139 "input": [
@@ -282,7 +282,7 @@
282 282 "shipped_days": 14
283 283 }
284 284 ],
285 − "expected": 7910.0
285 + "expected": 8022.0
286 286 },
287 287 {
288 288 "input": [
@@ -386,7 +386,7 @@
386 386 "shipped_days": 8
387 387 }
388 388 ],
389 − "expected": 4889.0
389 + "expected": 4951.0
390 390 },
391 391 {
392 392 "input": [
@@ -499,7 +499,7 @@
499 499 "shipped_days": 8
500 500 }
501 501 ],
502 − "expected": 4861.0
502 + "expected": 4973.0
503 503 },
504 504 {
505 505 "input": [
@@ -630,7 +630,7 @@
630 630 "shipped_days": 5
631 631 }
632 632 ],
633 − "expected": 7046.0
633 + "expected": 7147.0
634 634 }
635 635 ]
636 636 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00120_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00120_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "5b1b1089a793b0953579b8ce004efe99282ffd8a6eb17b16c3ce50273c121aad",
5 + "document_hash": "c5be33e5009123ba2c251884304b839ae6fc5163f0745c302cffbabf618b40cb",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 5, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6138: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 169 "shipped_days": 6
170 170 }
171 171 ],
172 − "expected": 11463.0
172 + "expected": 11919.2
173 173 },
174 174 {
175 175 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 8
229 229 }
230 230 ],
231 − "expected": 3698.4
231 + "expected": 3926.1
232 232 },
233 233 {
234 234 "input": [
@@ -323,7 +323,7 @@
323 323 "shipped_days": 1
324 324 }
325 325 ],
326 − "expected": 7350.4
326 + "expected": 7625.0
327 327 },
328 328 {
329 329 "input": [
@@ -499,7 +499,7 @@
499 499 "shipped_days": 10
500 500 }
501 501 ],
502 − "expected": 11559.0
502 + "expected": 12015.2
503 503 },
504 504 {
505 505 "input": [
@@ -666,7 +666,7 @@
666 666 "shipped_days": 10
667 667 }
668 668 ],
669 − "expected": 8903.5
669 + "expected": 9203.5
670 670 }
671 671 ]
672 672 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00121_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00121_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "9064e893e331010c27db6e1229c2af7da83ed7fb3680e079fc5c1a9ee7b2d73a",
5 + "document_hash": "1645df3901b5bd61ea8c6790e15a695b1144eef69bc075121ffd0c83e6fc13af",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6726: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 106 "shipped_days": 3
107 107 }
108 108 ],
109 − "expected": 2869.2
109 + "expected": 2888.6
110 110 },
111 111 {
112 112 "input": [
@@ -147,7 +147,7 @@
147 147 "shipped_days": 12
148 148 }
149 149 ],
150 − "expected": 3908.9
150 + "expected": 4089.0
151 151 },
152 152 {
153 153 "input": [
@@ -287,7 +287,7 @@
287 287 "shipped_days": 5
288 288 }
289 289 ],
290 − "expected": 6986.1
290 + "expected": 7212.7
291 291 },
292 292 {
293 293 "input": [
@@ -409,7 +409,7 @@
409 409 "shipped_days": 10
410 410 }
411 411 ],
412 − "expected": 8197.2
412 + "expected": 8385.9
413 413 },
414 414 {
415 415 "input": [
@@ -504,7 +504,7 @@
504 504 "shipped_days": 4
505 505 }
506 506 ],
507 − "expected": 6209.1
507 + "expected": 6372.1
508 508 }
509 509 ]
510 510 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00122_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00122_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "8641f3a09b033f2a216948ea197682c280cda7a9559b7102473da5f152a73eb3",
5 + "document_hash": "bd26e436a86876d04961cc142588671db4ead753f838933bde6907da5354f70e",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8821: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -142,7 +142,7 @@
142 142 "shipped_days": 8
143 143 }
144 144 ],
145 − "expected": 6993.4
145 + "expected": 7221.2
146 146 },
147 147 {
148 148 "input": [
@@ -345,7 +345,7 @@
345 345 "shipped_days": 7
346 346 }
347 347 ],
348 − "expected": 12417.7
348 + "expected": 12645.5
349 349 },
350 350 {
351 351 "input": [
@@ -440,7 +440,7 @@
440 440 "shipped_days": 4
441 441 }
442 442 ],
443 − "expected": 4310.8
443 + "expected": 4400.2
444 444 },
445 445 {
446 446 "input": [
@@ -535,7 +535,7 @@
535 535 "shipped_days": 8
536 536 }
537 537 ],
538 − "expected": 7689.7
538 + "expected": 7967.8
539 539 },
540 540 {
541 541 "input": [
@@ -576,7 +576,7 @@
576 576 "shipped_days": 12
577 577 }
578 578 ],
579 − "expected": 572.2
579 + "expected": 609.3
580 580 }
581 581 ]
582 582 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00123_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00123_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "3a8a85801b1200bb0a18063e20eb67097e3e9c34d009894a5a61963f7145608b",
5 + "document_hash": "6c868c5916de1b984d70dc327ee458814726f8ea6a1fad99f8c2fb35e7e77b7f",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6186: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 758
30 + "char_end": 631
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 160 "shipped_days": 9
161 161 }
162 162 ],
163 − "expected": 10163.85
163 + "expected": 10350.34
164 164 },
165 165 {
166 166 "input": [
@@ -318,7 +318,7 @@
318 318 "shipped_days": 1
319 319 }
320 320 ],
321 − "expected": 12367.03
321 + "expected": 12553.52
322 322 },
323 323 {
324 324 "input": [
@@ -404,7 +404,7 @@
404 404 "shipped_days": 8
405 405 }
406 406 ],
407 − "expected": 8167.73
407 + "expected": 8263.67
408 408 },
409 409 {
410 410 "input": [
@@ -463,7 +463,7 @@
463 463 "shipped_days": 1
464 464 }
465 465 ],
466 − "expected": 5503.85
466 + "expected": 5554.96
467 467 },
468 468 {
469 469 "input": [
@@ -603,7 +603,7 @@
603 603 "shipped_days": 11
604 604 }
605 605 ],
606 − "expected": 11804.57
606 + "expected": 11978.79
607 607 }
608 608 ]
609 609 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00124_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00124_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "adfa94aa242540ae900f58c2db73dace6e0bb6ec243a71137685b2d527e54fdb",
5 + "document_hash": "a285cd3f7f5bbfcaa508697bf261341dc847483a97e354b95faf09bc23bb626f",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4820: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 97 "shipped_days": 12
98 98 }
99 99 ],
100 − "expected": 5443.0
100 + "expected": 5641.0
101 101 },
102 102 {
103 103 "input": [
@@ -147,7 +147,7 @@
147 147 "shipped_days": 5
148 148 }
149 149 ],
150 − "expected": 5054.0
150 + "expected": 5237.0
151 151 },
152 152 {
153 153 "input": [
@@ -206,7 +206,7 @@
206 206 "shipped_days": 12
207 207 }
208 208 ],
209 − "expected": 3318.0
209 + "expected": 3387.0
210 210 },
211 211 {
212 212 "input": [
@@ -283,7 +283,7 @@
283 283 "shipped_days": 11
284 284 }
285 285 ],
286 − "expected": 3199.0
286 + "expected": 3369.0
287 287 },
288 288 {
289 289 "input": [
@@ -369,7 +369,7 @@
369 369 "shipped_days": 9
370 370 }
371 371 ],
372 − "expected": 4883.0
372 + "expected": 5156.0
373 373 }
374 374 ]
375 375 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00125_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00125_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "144a4af055228fc4045ad15b1ca9ba8229c8e5563cc4c4cf9ad94203a0072dd9",
5 + "document_hash": "386fcaff7daca426267d33af6f30657f21e4863b66d5a71d38d0b78a334eb069",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 1, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8156: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 97 "shipped_days": 13
98 98 }
99 99 ],
100 − "expected": 4817.6
100 + "expected": 4843.9
101 101 },
102 102 {
103 103 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 5
229 229 }
230 230 ],
231 − "expected": 7490.0
231 + "expected": 7655.4
232 232 },
233 233 {
234 234 "input": [
@@ -269,7 +269,7 @@
269 269 "shipped_days": 12
270 270 }
271 271 ],
272 − "expected": 2071.8
272 + "expected": 2098.1
273 273 },
274 274 {
275 275 "input": [
@@ -382,7 +382,7 @@
382 382 "shipped_days": 11
383 383 }
384 384 ],
385 − "expected": 8298.7
385 + "expected": 8541.9
386 386 },
387 387 {
388 388 "input": [
@@ -450,7 +450,7 @@
450 450 "shipped_days": 13
451 451 }
452 452 ],
453 − "expected": 5259.9
453 + "expected": 5536.5
454 454 }
455 455 ]
456 456 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00126_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00126_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "a6bed1ca1c6bee001ded092712010d4dc8d3ec10d0d1f85432e8844be28b9359",
5 + "document_hash": "0729452f4b76415c15bdd0075e98c4db16464515a675ea3d8e3611adb3c8a3e9",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1488: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 6
125 125 }
126 126 ],
127 − "expected": 5475.93
127 + "expected": 5529.1
128 128 },
129 129 {
130 130 "input": [
@@ -264,7 +264,7 @@
264 264 "shipped_days": 2
265 265 }
266 266 ],
267 − "expected": 11668.86
267 + "expected": 11941.25
268 268 },
269 269 {
270 270 "input": [
@@ -413,7 +413,7 @@
413 413 "shipped_days": 1
414 414 }
415 415 ],
416 − "expected": 11668.86
416 + "expected": 11941.25
417 417 },
418 418 {
419 419 "input": [
@@ -517,7 +517,7 @@
517 517 "shipped_days": 10
518 518 }
519 519 ],
520 − "expected": 8587.23
520 + "expected": 8696.85
521 521 },
522 522 {
523 523 "input": [
@@ -630,7 +630,7 @@
630 630 "shipped_days": 6
631 631 }
632 632 ],
633 − "expected": 8404.43
633 + "expected": 8676.83
634 634 }
635 635 ]
636 636 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00127_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00127_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "01e9b8b715cbfdd0d66c26d571044c18f9a6d9d2c5146b834291716879cf02d2",
5 + "document_hash": "bb6e6467042a1b7809f3ea38b83ea82e5554ca36b70042023f682efd5bae752a",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4907: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 14
134 134 }
135 135 ],
136 − "expected": 3326.0
136 + "expected": 3510.0
137 137 },
138 138 {
139 139 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 14
229 229 }
230 230 ],
231 − "expected": 283.0
231 + "expected": 296.0
232 232 },
233 233 {
234 234 "input": [
@@ -278,7 +278,7 @@
278 278 "shipped_days": 14
279 279 }
280 280 ],
281 − "expected": 2936.0
281 + "expected": 3097.0
282 282 },
283 283 {
284 284 "input": [
@@ -328,7 +328,7 @@
328 328 "shipped_days": 14
329 329 }
330 330 ],
331 − "expected": 355.0
331 + "expected": 376.0
332 332 },
333 333 {
334 334 "input": [
@@ -369,7 +369,7 @@
369 369 "shipped_days": 12
370 370 }
371 371 ],
372 − "expected": 1798.0
372 + "expected": 1896.0
373 373 }
374 374 ]
375 375 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00128_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00128_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "e702731f232f4b91d0513add4b301fd1ef0382eee35df22a98cef7eb727fafca",
5 + "document_hash": "ea7fe68afa60ebc43cd0a501c243c72cd3b993bfaef6e389f12308e1967e3471",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7370: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 761
30 + "char_end": 634
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -205,7 +205,7 @@
205 205 "shipped_days": 6
206 206 }
207 207 ],
208 − "expected": 8219.07
208 + "expected": 8595.95
209 209 },
210 210 {
211 211 "input": [
@@ -291,7 +291,7 @@
291 291 "shipped_days": 8
292 292 }
293 293 ],
294 − "expected": 4517.31
294 + "expected": 4718.05
295 295 },
296 296 {
297 297 "input": [
@@ -350,7 +350,7 @@
350 350 "shipped_days": 13
351 351 }
352 352 ],
353 − "expected": 2948.13
353 + "expected": 3062.55
354 354 },
355 355 {
356 356 "input": [
@@ -463,7 +463,7 @@
463 463 "shipped_days": 8
464 464 }
465 465 ],
466 − "expected": 4934.84
466 + "expected": 5211.5
467 467 },
468 468 {
469 469 "input": [
@@ -531,7 +531,7 @@
531 531 "shipped_days": 14
532 532 }
533 533 ],
534 − "expected": 3125.64
534 + "expected": 3300.81
535 535 }
536 536 ]
537 537 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00129_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00129_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "e508132470ec83c8745c1899d6c954b865130cad33382718fa48d35d6bea0ec8",
5 + "document_hash": "46f3b799f18b08ee443481845e8695eff10ac2f2b23c2b177eafce3b42e2ea40",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2059: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 2
89 89 }
90 90 ],
91 − "expected": 3878.09
91 + "expected": 3976.88
92 92 },
93 93 {
94 94 "input": [
@@ -129,7 +129,7 @@
129 129 "shipped_days": 2
130 130 }
131 131 ],
132 − "expected": 2759.16
132 + "expected": 2800.76
133 133 },
134 134 {
135 135 "input": [
@@ -188,7 +188,7 @@
188 188 "shipped_days": 6
189 189 }
190 190 ],
191 − "expected": 3993.65
191 + "expected": 4137.46
192 192 },
193 193 {
194 194 "input": [
@@ -346,7 +346,7 @@
346 346 "shipped_days": 14
347 347 }
348 348 ],
349 − "expected": 8439.66
349 + "expected": 8653.36
350 350 },
351 351 {
352 352 "input": [
@@ -549,7 +549,7 @@
549 549 "shipped_days": 1
550 550 }
551 551 ],
552 − "expected": 10565.94
552 + "expected": 10818.88
553 553 }
554 554 ]
555 555 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00130_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00130_2000",
3 3 "family": "exec",
4 4 "split": "dev",
5 − "document_hash": "c65f14fbcb1d4d0049a6e641be2826817637866851032f95c4cc1943f87ff809",
5 + "document_hash": "c415053d25119b89bd769d02be02c390efebcd68b992b946ddf043ad29041f86",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7512: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 6
89 89 }
90 90 ],
91 − "expected": 3609.6
91 + "expected": 3633.3
92 92 },
93 93 {
94 94 "input": [
@@ -188,7 +188,7 @@
188 188 "shipped_days": 2
189 189 }
190 190 ],
191 − "expected": 3823.2
191 + "expected": 3828.8
192 192 },
193 193 {
194 194 "input": [
@@ -301,7 +301,7 @@
301 301 "shipped_days": 13
302 302 }
303 303 ],
304 − "expected": 5885.5
304 + "expected": 5944.5
305 305 },
306 306 {
307 307 "input": [
@@ -378,7 +378,7 @@
378 378 "shipped_days": 6
379 379 }
380 380 ],
381 − "expected": 981.7
381 + "expected": 1035.1
382 382 }
383 383 ]
384 384 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00201_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00201_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "876f090ca1801217ec4295964e7d9ce5b46b9fc96b04db7ca463d6cc8ed74978",
5 + "document_hash": "065b6c18bbd16f16126924a54946ca4bf95c5863e57b14f33e4c9d88568f8549",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 4, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8684: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 115 "shipped_days": 8
116 116 }
117 117 ],
118 − "expected": 2996.0
118 + "expected": 3119.0
119 119 },
120 120 {
121 121 "input": [
@@ -264,7 +264,7 @@
264 264 "shipped_days": 1
265 265 }
266 266 ],
267 − "expected": 4291.0
267 + "expected": 4392.0
268 268 },
269 269 {
270 270 "input": [
@@ -323,7 +323,7 @@
323 323 "shipped_days": 10
324 324 }
325 325 ],
326 − "expected": 3088.0
326 + "expected": 3164.0
327 327 },
328 328 {
329 329 "input": [
@@ -373,7 +373,7 @@
373 373 "shipped_days": 8
374 374 }
375 375 ],
376 − "expected": 1229.0
376 + "expected": 1263.0
377 377 },
378 378 {
379 379 "input": [
@@ -540,7 +540,7 @@
540 540 "shipped_days": 7
541 541 }
542 542 ],
543 − "expected": 4869.0
543 + "expected": 4995.0
544 544 }
545 545 ]
546 546 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00202_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00202_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "660fc77600d7958889cef9b34f9c9a1a8c2d96ef38db054d180f225ebe3d4a42",
5 + "document_hash": "ec13194237af7efe7765a21f1d8303ab8ac449271eb48e3395906648c770bab8",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1160: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 749
30 + "char_end": 622
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 6
89 89 }
90 90 ],
91 − "expected": 5253.0
91 + "expected": 5495.0
92 92 },
93 93 {
94 94 "input": [
@@ -219,7 +219,7 @@
219 219 "shipped_days": 7
220 220 }
221 221 ],
222 − "expected": 9510.0
222 + "expected": 9809.0
223 223 },
224 224 {
225 225 "input": [
@@ -314,7 +314,7 @@
314 314 "shipped_days": 10
315 315 }
316 316 ],
317 − "expected": 4533.0
317 + "expected": 4631.0
318 318 },
319 319 {
320 320 "input": [
@@ -373,7 +373,7 @@
373 373 "shipped_days": 5
374 374 }
375 375 ],
376 − "expected": 3891.0
376 + "expected": 3976.0
377 377 },
378 378 {
379 379 "input": [
@@ -459,7 +459,7 @@
459 459 "shipped_days": 7
460 460 }
461 461 ],
462 − "expected": 3112.0
462 + "expected": 3118.0
463 463 }
464 464 ]
465 465 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00203_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00203_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "f26e049133c3f806d2045fa255ba4e6c2594f5ecbe872414b75faaf52ceb5033",
5 + "document_hash": "39eee37c7ba5fe1b6d8fce9eef4c00e10a33d8ca049d12ae01b9521d4ac60863",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-7417: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -142,7 +142,7 @@
142 142 "shipped_days": 6
143 143 }
144 144 ],
145 − "expected": 6045.2
145 + "expected": 6107.6
146 146 },
147 147 {
148 148 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 10
229 229 }
230 230 ],
231 − "expected": 5428.4
231 + "expected": 5488.4
232 232 },
233 233 {
234 234 "input": [
@@ -269,7 +269,7 @@
269 269 "shipped_days": 5
270 270 }
271 271 ],
272 − "expected": 1727.0
272 + "expected": 1748.7
273 273 },
274 274 {
275 275 "input": [
@@ -373,7 +373,7 @@
373 373 "shipped_days": 5
374 374 }
375 375 ],
376 − "expected": 5395.2
376 + "expected": 5419.4
377 377 },
378 378 {
379 379 "input": [
@@ -468,7 +468,7 @@
468 468 "shipped_days": 14
469 469 }
470 470 ],
471 − "expected": 4819.8
471 + "expected": 4844.0
472 472 }
473 473 ]
474 474 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00204_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00204_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "7cc296695d97fdf8fb07de5772744af9c3a79a52f20ee1e3410187cac4dd94b2",
5 + "document_hash": "0aaf1a095160aa21c4485db3b1b171e86d9d41b904ac2a2ea102cc13df19cd22",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 2, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2345: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 17 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -151,7 +151,7 @@
151 151 "shipped_days": 14
152 152 }
153 153 ],
154 − "expected": 9785.1
154 + "expected": 10092.0
155 155 },
156 156 {
157 157 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 5
229 229 }
230 230 ],
231 − "expected": 5732.4
231 + "expected": 5927.4
232 232 },
233 233 {
234 234 "input": [
@@ -341,7 +341,7 @@
341 341 "shipped_days": 11
342 342 }
343 343 ],
344 − "expected": 4080.5
344 + "expected": 4275.4
345 345 },
346 346 {
347 347 "input": [
@@ -382,7 +382,7 @@
382 382 "shipped_days": 14
383 383 }
384 384 ],
385 − "expected": 5240.0
385 + "expected": 5338.3
386 386 },
387 387 {
388 388 "input": [
modified data/generated/keys/exec/exec_00205_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00205_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "1f9b534774ca095126415c70bc29f4400d55f083ec020a658e6ee1263a76072f",
5 + "document_hash": "c82f149ddbd77be80773d103825621fb187f06bb60d4fd570c73c2b3cd898ecb",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9090: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 169 "shipped_days": 13
170 170 }
171 171 ],
172 − "expected": 6545.6
172 + "expected": 6710.2
173 173 },
174 174 {
175 175 "input": [
@@ -345,7 +345,7 @@
345 345 "shipped_days": 3
346 346 }
347 347 ],
348 − "expected": 9238.4
348 + "expected": 9422.1
349 349 },
350 350 {
351 351 "input": [
@@ -557,7 +557,7 @@
557 557 "shipped_days": 6
558 558 }
559 559 ],
560 − "expected": 10107.0
560 + "expected": 10290.7
561 561 },
562 562 {
563 563 "input": [
@@ -751,7 +751,7 @@
751 751 "shipped_days": 4
752 752 }
753 753 ],
754 − "expected": 10107.0
754 + "expected": 10290.7
755 755 },
756 756 {
757 757 "input": [
@@ -864,7 +864,7 @@
864 864 "shipped_days": 5
865 865 }
866 866 ],
867 − "expected": 5911.0
867 + "expected": 5989.7
868 868 }
869 869 ]
870 870 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00206_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00206_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "71e5bf98eb246c53babd99cdd8ec1212e594a1af3ddd2a6da69bac721ad65ddd",
5 + "document_hash": "e391c9d757bc03677c664222c3ff3f1bbab881d7f566c0072b184e0b90503339",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 4, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 759
30 + "char_end": 632
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 13
125 125 }
126 126 ],
127 − "expected": 9100.4
127 + "expected": 9357.5
128 128 },
129 129 {
130 130 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 13
229 229 }
230 230 ],
231 − "expected": 9506.1
231 + "expected": 9871.6
232 232 },
233 233 {
234 234 "input": [
@@ -314,7 +314,7 @@
314 314 "shipped_days": 5
315 315 }
316 316 ],
317 − "expected": 10907.8
317 + "expected": 11222.5
318 318 },
319 319 {
320 320 "input": [
@@ -364,7 +364,7 @@
364 364 "shipped_days": 13
365 365 }
366 366 ],
367 − "expected": 4655.5
367 + "expected": 4727.9
368 368 },
369 369 {
370 370 "input": [
@@ -405,7 +405,7 @@
405 405 "shipped_days": 9
406 406 }
407 407 ],
408 − "expected": 3240.6
408 + "expected": 3352.1
409 409 }
410 410 ]
411 411 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00207_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00207_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "2273befa734c54f0d780db1126ec3ce69ce8f9f829e57586a614786258681ba3",
5 + "document_hash": "7d024469b5d38776a899aa870f5af7e146953a19f1adc1d91f634cda62aa4c14",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-3534: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 7
134 134 }
135 135 ],
136 − "expected": 6347.24
136 + "expected": 6534.69
137 137 },
138 138 {
139 139 "input": [
@@ -237,7 +237,7 @@
237 237 "shipped_days": 2
238 238 }
239 239 ],
240 − "expected": 4633.07
240 + "expected": 4793.52
241 241 },
242 242 {
243 243 "input": [
@@ -413,7 +413,7 @@
413 413 "shipped_days": 11
414 414 }
415 415 ],
416 − "expected": 10472.88
416 + "expected": 10704.58
417 417 },
418 418 {
419 419 "input": [
@@ -634,7 +634,7 @@
634 634 "shipped_days": 14
635 635 }
636 636 ],
637 − "expected": 15420.92
637 + "expected": 15953.22
638 638 },
639 639 {
640 640 "input": [
@@ -846,7 +846,7 @@
846 846 "shipped_days": 7
847 847 }
848 848 ],
849 − "expected": 15224.85
849 + "expected": 15746.83
850 850 }
851 851 ]
852 852 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00208_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00208_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "e22e990c3c20d7feee2f70e993961e465ac68d3a867c3547dc0566739847c2d8",
5 + "document_hash": "8528b3fafcf58b9daa88ffa9d7b9b25fb6e225f9bb767849744c05510a5ace37",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 2, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9673: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 760
30 + "char_end": 633
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 8
89 89 }
90 90 ],
91 − "expected": 6763.9
91 + "expected": 7103.9
92 92 },
93 93 {
94 94 "input": [
@@ -156,7 +156,7 @@
156 156 "shipped_days": 14
157 157 }
158 158 ],
159 − "expected": 4378.0
159 + "expected": 4589.7
160 160 },
161 161 {
162 162 "input": [
@@ -251,7 +251,7 @@
251 251 "shipped_days": 5
252 252 }
253 253 ],
254 − "expected": 8918.9
254 + "expected": 9380.2
255 255 },
256 256 {
257 257 "input": [
@@ -292,7 +292,7 @@
292 292 "shipped_days": 6
293 293 }
294 294 ],
295 − "expected": 279.1
295 + "expected": 297.1
296 296 },
297 297 {
298 298 "input": [
@@ -441,7 +441,7 @@
441 441 "shipped_days": 9
442 442 }
443 443 ],
444 − "expected": 12672.3
444 + "expected": 13279.3
445 445 }
446 446 ]
447 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00209_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00209_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "bd895760d266f605848302775beaceb129633108ddc74dcbe83a8642eecb0ea3",
5 + "document_hash": "27a0f4409b3611173adf72865b52c6beef014bce0282fd0498be50deeed192c9",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2760: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 7
89 89 }
90 90 ],
91 − "expected": 5042.6
91 + "expected": 5250.3
92 92 },
93 93 {
94 94 "input": [
@@ -219,7 +219,7 @@
219 219 "shipped_days": 12
220 220 }
221 221 ],
222 − "expected": 7168.3
222 + "expected": 7499.5
223 223 },
224 224 {
225 225 "input": [
@@ -305,7 +305,7 @@
305 305 "shipped_days": 14
306 306 }
307 307 ],
308 − "expected": 4988.2
308 + "expected": 5188.4
309 309 },
310 310 {
311 311 "input": [
@@ -436,7 +436,7 @@
436 436 "shipped_days": 9
437 437 }
438 438 ],
439 − "expected": 7168.3
439 + "expected": 7499.5
440 440 },
441 441 {
442 442 "input": [
@@ -531,7 +531,7 @@
531 531 "shipped_days": 7
532 532 }
533 533 ],
534 − "expected": 3551.1
534 + "expected": 3674.6
535 535 }
536 536 ]
537 537 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00210_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00210_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "2678fb8184c3c22ce8bfa861af1a68ef2704dd84cb88718d63c1b668adaf1f69",
5 + "document_hash": "ffa82307f9b4986f3fc473b8b7df1e4f8f0e1c25d3b6e36cdcf4114a763427c5",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6939: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 5 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 14
89 89 }
90 90 ],
91 − "expected": 4584.59
91 + "expected": 4701.32
92 92 },
93 93 {
94 94 "input": [
@@ -174,7 +174,7 @@
174 174 "shipped_days": 4
175 175 }
176 176 ],
177 − "expected": 5338.5
177 + "expected": 5502.36
178 178 },
179 179 {
180 180 "input": [
@@ -296,7 +296,7 @@
296 296 "shipped_days": 8
297 297 }
298 298 ],
299 − "expected": 7430.25
299 + "expected": 7634.54
300 300 },
301 301 {
302 302 "input": [
@@ -427,7 +427,7 @@
427 427 "shipped_days": 12
428 428 }
429 429 ],
430 − "expected": 8724.28
430 + "expected": 8928.56
431 431 },
432 432 {
433 433 "input": [
@@ -495,7 +495,7 @@
495 495 "shipped_days": 8
496 496 }
497 497 ],
498 − "expected": 3752.34
498 + "expected": 3916.2
499 499 }
500 500 ]
501 501 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00211_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00211_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "5ad4fd06e4149025eddafab6fbf5c707e06c1f7e47f3cbaec839f52cd6744e38",
5 + "document_hash": "95faa51b0302db54273e9b4469b6eae92c251463238fdad494d437435c293548",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6353: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 753
30 + "char_end": 626
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -169,7 +169,7 @@
169 169 "shipped_days": 12
170 170 }
171 171 ],
172 − "expected": 6694.0
172 + "expected": 6816.0
173 173 },
174 174 {
175 175 "input": [
@@ -210,7 +210,7 @@
210 210 "shipped_days": 5
211 211 }
212 212 ],
213 − "expected": 2616.0
213 + "expected": 2665.0
214 214 },
215 215 {
216 216 "input": [
@@ -314,7 +314,7 @@
314 314 "shipped_days": 1
315 315 }
316 316 ],
317 − "expected": 5873.0
317 + "expected": 5987.0
318 318 },
319 319 {
320 320 "input": [
@@ -508,7 +508,7 @@
508 508 "shipped_days": 1
509 509 }
510 510 ],
511 − "expected": 12003.0
511 + "expected": 12354.0
512 512 },
513 513 {
514 514 "input": [
modified data/generated/keys/exec/exec_00212_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00212_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "8a171bf76e4dc9c8921e9b2a4370fc5385ceb9695d62d19fc693110c84df06f0",
5 + "document_hash": "2be5dc8496484a99fc11a00be474f4f7f139a8f491a2ebf3fff0db8e7804dd82",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 3, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8815: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 753
30 + "char_end": 626
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -70,7 +70,7 @@
70 70 "shipped_days": 6
71 71 }
72 72 ],
73 − "expected": 1807.0
73 + "expected": 1814.0
74 74 },
75 75 {
76 76 "input": [
@@ -156,7 +156,7 @@
156 156 "shipped_days": 13
157 157 }
158 158 ],
159 − "expected": 6185.0
159 + "expected": 6313.0
160 160 },
161 161 {
162 162 "input": [
@@ -359,7 +359,7 @@
359 359 "shipped_days": 6
360 360 }
361 361 ],
362 − "expected": 12224.0
362 + "expected": 12499.0
363 363 },
364 364 {
365 365 "input": [
@@ -571,7 +571,7 @@
571 571 "shipped_days": 10
572 572 }
573 573 ],
574 − "expected": 12154.0
574 + "expected": 12429.0
575 575 },
576 576 {
577 577 "input": [
@@ -738,7 +738,7 @@
738 738 "shipped_days": 3
739 739 }
740 740 ],
741 − "expected": 9732.0
741 + "expected": 10002.0
742 742 }
743 743 ]
744 744 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00213_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00213_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "55fc33fe81ccfe25e3ebdcdb70ee135b990fcf94d02bfb56e0116982ffd6df6f",
5 + "document_hash": "3c65c42333c25f1f3f90331747f08fafce263c8faa11a56d3a68bc58b2217327",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 4, \"excl_region\": \"Nord\", \"rounding\": 2, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9971: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 26 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 758
30 + "char_end": 631
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 14
125 125 }
126 126 ],
127 − "expected": 7666.09
127 + "expected": 7921.14
128 128 },
129 129 {
130 130 "input": [
@@ -210,7 +210,7 @@
210 210 "shipped_days": 11
211 211 }
212 212 ],
213 − "expected": 3462.69
213 + "expected": 3617.44
214 214 },
215 215 {
216 216 "input": [
@@ -341,7 +341,7 @@
341 341 "shipped_days": 4
342 342 }
343 343 ],
344 − "expected": 6351.6
344 + "expected": 6615.95
345 345 },
346 346 {
347 347 "input": [
@@ -418,7 +418,7 @@
418 418 "shipped_days": 13
419 419 }
420 420 ],
421 − "expected": 4539.1
421 + "expected": 4803.45
422 422 },
423 423 {
424 424 "input": [
@@ -495,7 +495,7 @@
495 495 "shipped_days": 9
496 496 }
497 497 ],
498 − "expected": 2094.96
498 + "expected": 2181.25
499 499 }
500 500 ]
501 501 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00214_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00214_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "0be81f6d80f5eb03ba862cbc94c8bf20b6ab1196483c947401585e7178fd458d",
5 + "document_hash": "ac0a3ee69ec7da626616f6ffb5e443a4f26a3d7857caaa20f1d4ebfa5355d28b",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6115: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 6
134 134 }
135 135 ],
136 − "expected": 9188.3
136 + "expected": 9669.7
137 137 },
138 138 {
139 139 "input": [
@@ -264,7 +264,7 @@
264 264 "shipped_days": 13
265 265 }
266 266 ],
267 − "expected": 10735.2
267 + "expected": 11230.4
268 268 },
269 269 {
270 270 "input": [
@@ -368,7 +368,7 @@
368 368 "shipped_days": 10
369 369 }
370 370 ],
371 − "expected": 6277.4
371 + "expected": 6401.6
372 372 },
373 373 {
374 374 "input": [
@@ -418,7 +418,7 @@
418 418 "shipped_days": 4
419 419 }
420 420 ],
421 − "expected": 6555.5
421 + "expected": 6789.8
422 422 },
423 423 {
424 424 "input": [
@@ -486,7 +486,7 @@
486 486 "shipped_days": 14
487 487 }
488 488 ],
489 − "expected": 7305.1
489 + "expected": 7485.3
490 490 }
491 491 ]
492 492 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00215_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00215_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "37b95a047621c2a78beaff095b27647064c994bf6ede6ef6b3e91f8296b70eff",
5 + "document_hash": "a4094c70181359b8442ead384be9497e3fb0c46a4d9266cce07ca77bd9e26e2b",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 5, \"min_units\": 2, \"excl_region\": \"Centre\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9778: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 6 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 2 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 753
30 + "char_end": 626
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 2
125 125 }
126 126 ],
127 − "expected": 7073.4
127 + "expected": 7469.6
128 128 },
129 129 {
130 130 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 9
229 229 }
230 230 ],
231 − "expected": 12236.6
231 + "expected": 12753.4
232 232 },
233 233 {
234 234 "input": [
@@ -431,7 +431,7 @@
431 431 "shipped_days": 6
432 432 }
433 433 ],
434 − "expected": 19490.8
434 + "expected": 20372.7
435 435 },
436 436 {
437 437 "input": [
@@ -598,7 +598,7 @@
598 598 "shipped_days": 5
599 599 }
600 600 ],
601 − "expected": 15129.1
601 + "expected": 15765.2
602 602 },
603 603 {
604 604 "input": [
@@ -729,7 +729,7 @@
729 729 "shipped_days": 8
730 730 }
731 731 ],
732 − "expected": 13158.9
732 + "expected": 13686.7
733 733 }
734 734 ]
735 735 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00216_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00216_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "1deddf301b0b2291cb9bdc4213848eceb17ece12032a2e454bcceb4ca4174397",
5 + "document_hash": "1a218de32e6fe4bbe97927ea16330bda3b598ed66c1517824d94ff90f944e802",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-5343: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 756
30 + "char_end": 629
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -142,7 +142,7 @@
142 142 "shipped_days": 9
143 143 }
144 144 ],
145 − "expected": 5896.65
145 + "expected": 6072.0
146 146 },
147 147 {
148 148 "input": [
@@ -264,7 +264,7 @@
264 264 "shipped_days": 6
265 265 }
266 266 ],
267 − "expected": 7365.6
267 + "expected": 7600.1
268 268 },
269 269 {
270 270 "input": [
@@ -458,7 +458,7 @@
458 458 "shipped_days": 10
459 459 }
460 460 ],
461 − "expected": 8732.34
461 + "expected": 9064.48
462 462 },
463 463 {
464 464 "input": [
@@ -526,7 +526,7 @@
526 526 "shipped_days": 14
527 527 }
528 528 ],
529 − "expected": 2062.77
529 + "expected": 2197.03
530 530 },
531 531 {
532 532 "input": [
@@ -711,7 +711,7 @@
711 711 "shipped_days": 10
712 712 }
713 713 ],
714 − "expected": 8732.34
714 + "expected": 9064.48
715 715 }
716 716 ]
717 717 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00217_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00217_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "7f71cb12d994e6fe2646a0f7b337c441f923868ec6b21c29533122fda4d87a73",
5 + "document_hash": "1223c7391b1da1aeae2d242c0e41de4639218cfe0c90ba9065f1cce42f0eedbd",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1928: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 20 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 753
30 + "char_end": 626
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 10
134 134 }
135 135 ],
136 − "expected": 6373.0
136 + "expected": 6665.0
137 137 },
138 138 {
139 139 "input": [
@@ -201,7 +201,7 @@
201 201 "shipped_days": 13
202 202 }
203 203 ],
204 − "expected": 4341.0
204 + "expected": 4602.0
205 205 },
206 206 {
207 207 "input": [
@@ -350,7 +350,7 @@
350 350 "shipped_days": 11
351 351 }
352 352 ],
353 − "expected": 11700.0
353 + "expected": 12228.0
354 354 },
355 355 {
356 356 "input": [
@@ -562,7 +562,7 @@
562 562 "shipped_days": 5
563 563 }
564 564 ],
565 − "expected": 16809.0
565 + "expected": 17574.0
566 566 },
567 567 {
568 568 "input": [
@@ -693,7 +693,7 @@
693 693 "shipped_days": 4
694 694 }
695 695 ],
696 − "expected": 9406.0
696 + "expected": 9766.0
697 697 }
698 698 ]
699 699 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00218_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00218_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "2a77fd38142317b7e8868dfcaebb42248ddd462e9339268378a26e4b587eab3d",
5 + "document_hash": "971d94c4e7b4ae02142ffb6eb92af9d2cc0f6e4d52e94a64c5132b510197ce8a",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6421: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 22 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 752
30 + "char_end": 625
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -187,7 +187,7 @@
187 187 "shipped_days": 9
188 188 }
189 189 ],
190 − "expected": 8482.0
190 + "expected": 8709.0
191 191 },
192 192 {
193 193 "input": [
@@ -255,7 +255,7 @@
255 255 "shipped_days": 9
256 256 }
257 257 ],
258 − "expected": 1995.0
258 + "expected": 2060.0
259 259 },
260 260 {
261 261 "input": [
@@ -350,7 +350,7 @@
350 350 "shipped_days": 3
351 351 }
352 352 ],
353 − "expected": 3350.0
353 + "expected": 3392.0
354 354 },
355 355 {
356 356 "input": [
@@ -490,7 +490,7 @@
490 490 "shipped_days": 6
491 491 }
492 492 ],
493 − "expected": 5406.0
493 + "expected": 5568.0
494 494 },
495 495 {
496 496 "input": [
@@ -648,7 +648,7 @@
648 648 "shipped_days": 10
649 649 }
650 650 ],
651 − "expected": 8482.0
651 + "expected": 8709.0
652 652 }
653 653 ]
654 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00219_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00219_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "501d502ac1cd6920ddaeccbc4f0c925ddf4ca966cfb601ffd9d0547d97a741ed",
5 + "document_hash": "390edd848add1a4cf011660e8ac9a6a6aa381c4508a8f68635f3217ce20979cb",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 5, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-1173: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 9 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 759
30 + "char_end": 632
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -187,7 +187,7 @@
187 187 "shipped_days": 12
188 188 }
189 189 ],
190 − "expected": 9319.14
190 + "expected": 9797.23
191 191 },
192 192 {
193 193 "input": [
@@ -381,7 +381,7 @@
381 381 "shipped_days": 11
382 382 }
383 383 ],
384 − "expected": 15430.66
384 + "expected": 16029.27
385 385 },
386 386 {
387 387 "input": [
@@ -503,7 +503,7 @@
503 503 "shipped_days": 14
504 504 }
505 505 ],
506 − "expected": 13562.0
506 + "expected": 14131.7
507 507 },
508 508 {
509 509 "input": [
@@ -652,7 +652,7 @@
652 652 "shipped_days": 6
653 653 }
654 654 ],
655 − "expected": 15200.05
655 + "expected": 15771.11
656 656 },
657 657 {
658 658 "input": [
@@ -711,7 +711,7 @@
711 711 "shipped_days": 6
712 712 }
713 713 ],
714 − "expected": 6985.93
714 + "expected": 7345.77
715 715 }
716 716 ]
717 717 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00220_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00220_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "a7487c2eb12039fcc44c64460641febdd19eaf0e13d8b6b8f00d018ca564a8b6",
5 + "document_hash": "31eca9e7afc70ac5a795edc1d1529c531e0d028342802a61f4496c013e13bf45",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6509: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 19 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 759
30 + "char_end": 632
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -205,7 +205,7 @@
205 205 "shipped_days": 12
206 206 }
207 207 ],
208 − "expected": 10109.6
208 + "expected": 10644.1
209 209 },
210 210 {
211 211 "input": [
@@ -309,7 +309,7 @@
309 309 "shipped_days": 3
310 310 }
311 311 ],
312 − "expected": 3859.8
312 + "expected": 4025.2
313 313 },
314 314 {
315 315 "input": [
@@ -449,7 +449,7 @@
449 449 "shipped_days": 3
450 450 }
451 451 ],
452 − "expected": 9322.5
452 + "expected": 9846.6
453 453 },
454 454 {
455 455 "input": [
@@ -526,7 +526,7 @@
526 526 "shipped_days": 1
527 527 }
528 528 ],
529 − "expected": 4911.7
529 + "expected": 5199.9
530 530 },
531 531 {
532 532 "input": [
@@ -639,7 +639,7 @@
639 639 "shipped_days": 8
640 640 }
641 641 ],
642 − "expected": 6462.4
642 + "expected": 6789.1
643 643 }
644 644 ]
645 645 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00221_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00221_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "a986eec8e76aa658fd694272657b8e201d940be0671e7769a934ffe76ca4748a",
5 + "document_hash": "e228dfaa0d04173ee28b43513e5fc06914edf5d0c02b09bf0fc534531a583669",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Centre\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8588: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 760
30 + "char_end": 633
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 115 "shipped_days": 10
116 116 }
117 117 ],
118 − "expected": 6742.98
118 + "expected": 6948.54
119 119 },
120 120 {
121 121 "input": [
@@ -165,7 +165,7 @@
165 165 "shipped_days": 5
166 166 }
167 167 ],
168 − "expected": 2437.55
168 + "expected": 2557.8
169 169 },
170 170 {
171 171 "input": [
@@ -287,7 +287,7 @@
287 287 "shipped_days": 10
288 288 }
289 289 ],
290 − "expected": 7289.29
290 + "expected": 7515.78
291 291 },
292 292 {
293 293 "input": [
@@ -355,7 +355,7 @@
355 355 "shipped_days": 5
356 356 }
357 357 ],
358 − "expected": 3623.37
358 + "expected": 3736.67
359 359 },
360 360 {
361 361 "input": [
@@ -423,7 +423,7 @@
423 423 "shipped_days": 8
424 424 }
425 425 ],
426 − "expected": 2195.08
426 + "expected": 2213.17
427 427 }
428 428 ]
429 429 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00222_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00222_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "fa31c738b8b80170524adedf048965eb739a7ebf7de9fcd4f4192d331dcc3346",
5 + "document_hash": "040260f25a42680fcc038386368506adaa004bc2592e75e3fbb5903b54cc19ab",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4818: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 21 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 750
30 + "char_end": 623
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -124,7 +124,7 @@
124 124 "shipped_days": 4
125 125 }
126 126 ],
127 − "expected": 6567.0
127 + "expected": 6895.0
128 128 },
129 129 {
130 130 "input": [
@@ -282,7 +282,7 @@
282 282 "shipped_days": 1
283 283 }
284 284 ],
285 − "expected": 11349.0
285 + "expected": 11712.0
286 286 },
287 287 {
288 288 "input": [
@@ -422,7 +422,7 @@
422 422 "shipped_days": 14
423 423 }
424 424 ],
425 − "expected": 10867.0
425 + "expected": 11226.0
426 426 },
427 427 {
428 428 "input": [
@@ -526,7 +526,7 @@
526 526 "shipped_days": 11
527 527 }
528 528 ],
529 − "expected": 5274.0
529 + "expected": 5511.0
530 530 },
531 531 {
532 532 "input": [
@@ -576,7 +576,7 @@
576 576 "shipped_days": 6
577 577 }
578 578 ],
579 − "expected": 1237.0
579 + "expected": 1268.0
580 580 }
581 581 ]
582 582 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00223_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00223_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "9b2d718a88e7b451a6e6d433e81fa6c3862777588f40791fe888b2d7a44efa42",
5 + "document_hash": "7eef13486249d89cf05bda367ea9dae2ce2479b2236c1917b2f067c49a981619",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4982: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -214,7 +214,7 @@
214 214 "shipped_days": 1
215 215 }
216 216 ],
217 − "expected": 12157.0
217 + "expected": 12469.0
218 218 },
219 219 {
220 220 "input": [
@@ -309,7 +309,7 @@
309 309 "shipped_days": 8
310 310 }
311 311 ],
312 − "expected": 3305.0
312 + "expected": 3371.0
313 313 },
314 314 {
315 315 "input": [
@@ -494,7 +494,7 @@
494 494 "shipped_days": 9
495 495 }
496 496 ],
497 − "expected": 10095.0
497 + "expected": 10407.0
498 498 },
499 499 {
500 500 "input": [
@@ -562,7 +562,7 @@
562 562 "shipped_days": 9
563 563 }
564 564 ],
565 − "expected": 5109.0
565 + "expected": 5263.0
566 566 },
567 567 {
568 568 "input": [
@@ -720,7 +720,7 @@
720 720 "shipped_days": 1
721 721 }
722 722 ],
723 − "expected": 10891.0
723 + "expected": 11167.0
724 724 }
725 725 ]
726 726 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00224_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00224_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "526a53c484a99dde8d83b8fc83d103060128d6331659920d7688b8836de1ec53",
5 + "document_hash": "3e642abb57841365b827053f2437a250dcb419539389820bac778d4596c064fd",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2924: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 15 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 760
30 + "char_end": 633
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 106 "shipped_days": 12
107 107 }
108 108 ],
109 − "expected": 5224.44
109 + "expected": 5450.26
110 110 },
111 111 {
112 112 "input": [
@@ -165,7 +165,7 @@
165 165 "shipped_days": 4
166 166 }
167 167 ],
168 − "expected": 1529.84
168 + "expected": 1612.2
169 169 },
170 170 {
171 171 "input": [
@@ -287,7 +287,7 @@
287 287 "shipped_days": 8
288 288 }
289 289 ],
290 − "expected": 7160.51
290 + "expected": 7528.74
291 291 },
292 292 {
293 293 "input": [
@@ -364,7 +364,7 @@
364 364 "shipped_days": 11
365 365 }
366 366 ],
367 − "expected": 3959.61
367 + "expected": 4149.28
368 368 },
369 369 {
370 370 "input": [
@@ -423,7 +423,7 @@
423 423 "shipped_days": 6
424 424 }
425 425 ],
426 − "expected": 2005.6
426 + "expected": 2086.3
427 427 }
428 428 ]
429 429 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00225_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00225_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "0d5d08bcef45a939605c15cb0d0b5b1ac3c2304c8358dae05fa7da75db07621f",
5 + "document_hash": "54918c16a4ae3706a87d182112fca2a39c82c77e3a60fd25a8b47b958f2a6b21",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Centre':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 6, \"excl_region\": \"Centre\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-3204: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 7 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Centre'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -79,7 +79,7 @@
79 79 "shipped_days": 13
80 80 }
81 81 ],
82 − "expected": 2122.0
82 + "expected": 2201.0
83 83 },
84 84 {
85 85 "input": [
@@ -174,7 +174,7 @@
174 174 "shipped_days": 10
175 175 }
176 176 ],
177 − "expected": 3922.0
177 + "expected": 4123.0
178 178 },
179 179 {
180 180 "input": [
@@ -251,7 +251,7 @@
251 251 "shipped_days": 12
252 252 }
253 253 ],
254 − "expected": 1266.0
254 + "expected": 1352.0
255 255 },
256 256 {
257 257 "input": [
@@ -319,7 +319,7 @@
319 319 "shipped_days": 1
320 320 }
321 321 ],
322 − "expected": 3208.0
322 + "expected": 3364.0
323 323 },
324 324 {
325 325 "input": [
@@ -414,7 +414,7 @@
414 414 "shipped_days": 13
415 415 }
416 416 ],
417 − "expected": 3922.0
417 + "expected": 4123.0
418 418 }
419 419 ]
420 420 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00226_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00226_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "32ca9297d301e1956917680ad370eda2add0304f9742c49a2023cfa3f2a82bd2",
5 + "document_hash": "7a73067df93b17aba1a608bce48f7a6ff16d480af8d23713ec889405f279fc7d",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 3, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9781: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 8 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 750
30 + "char_end": 623
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 106 "shipped_days": 9
107 107 }
108 108 ],
109 − "expected": 6307.0
109 + "expected": 6505.0
110 110 },
111 111 {
112 112 "input": [
@@ -156,7 +156,7 @@
156 156 "shipped_days": 7
157 157 }
158 158 ],
159 − "expected": 1138.0
159 + "expected": 1182.0
160 160 },
161 161 {
162 162 "input": [
@@ -224,7 +224,7 @@
224 224 "shipped_days": 3
225 225 }
226 226 ],
227 − "expected": 3931.0
227 + "expected": 4064.0
228 228 },
229 229 {
230 230 "input": [
@@ -337,7 +337,7 @@
337 337 "shipped_days": 3
338 338 }
339 339 ],
340 − "expected": 8074.0
340 + "expected": 8340.0
341 341 },
342 342 {
343 343 "input": [
@@ -486,7 +486,7 @@
486 486 "shipped_days": 14
487 487 }
488 488 ],
489 − "expected": 7468.0
489 + "expected": 7645.0
490 490 }
491 491 ]
492 492 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00227_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00227_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "0297d0253d8ecb7ba91156d4221aff1d54ba5efaed7fe7664734e63d44d31bd0",
5 + "document_hash": "6afe19e10d57be0082db797187045b004680fa22a13a56a7b8f7e2a51b40597d",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4863: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 751
30 + "char_end": 624
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -70,7 +70,7 @@
70 70 "shipped_days": 1
71 71 }
72 72 ],
73 − "expected": 3306.5
73 + "expected": 3366.2
74 74 },
75 75 {
76 76 "input": [
@@ -138,7 +138,7 @@
138 138 "shipped_days": 10
139 139 }
140 140 ],
141 − "expected": 5976.1
141 + "expected": 6181.3
142 142 },
143 143 {
144 144 "input": [
@@ -242,7 +242,7 @@
242 242 "shipped_days": 8
243 243 }
244 244 ],
245 − "expected": 5590.1
245 + "expected": 5863.7
246 246 },
247 247 {
248 248 "input": [
@@ -346,7 +346,7 @@
346 346 "shipped_days": 3
347 347 }
348 348 ],
349 − "expected": 7501.9
349 + "expected": 7775.4
350 350 },
351 351 {
352 352 "input": [
@@ -387,7 +387,7 @@
387 387 "shipped_days": 14
388 388 }
389 389 ],
390 − "expected": 4197.5
390 + "expected": 4333.5
391 391 }
392 392 ]
393 393 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00228_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00228_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "7bd8eb7ac7409f89d5f44f1f5001fd0660aece03d12d92588e821c86e0d1f69b",
5 + "document_hash": "08d5c40fa24cccbcccaac14d408c241ab602604a712b727f12f570eb56887a74",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 5 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 5, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 5 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-3241: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 751
30 + "char_end": 624
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 115 "shipped_days": 1
116 116 }
117 117 ],
118 − "expected": 4217.76
118 + "expected": 4336.63
119 119 },
120 120 {
121 121 "input": [
@@ -165,7 +165,7 @@
165 165 "shipped_days": 10
166 166 }
167 167 ],
168 − "expected": 1667.06
168 + "expected": 1740.67
169 169 },
170 170 {
171 171 "input": [
@@ -296,7 +296,7 @@
296 296 "shipped_days": 5
297 297 }
298 298 ],
299 − "expected": 6250.25
299 + "expected": 6433.51
300 300 },
301 301 {
302 302 "input": [
@@ -391,7 +391,7 @@
391 391 "shipped_days": 5
392 392 }
393 393 ],
394 − "expected": 6440.03
394 + "expected": 6612.64
395 395 },
396 396 {
397 397 "input": [
@@ -603,7 +603,7 @@
603 603 "shipped_days": 1
604 604 }
605 605 ],
606 − "expected": 10273.51
606 + "expected": 10576.4
607 607 }
608 608 ]
609 609 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00229_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00229_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "488c5efd60f8e64b88269226071b8befe058a5e2442b99934157b38f5fc41a19",
5 + "document_hash": "b6a34443a51bb95703036fc2467d0c50ad248f99f3c11c0a35d6bc7fcf0ad481",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 6, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2051: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 12 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 2
134 134 }
135 135 ],
136 − "expected": 7117.0
136 + "expected": 7480.0
137 137 },
138 138 {
139 139 "input": [
@@ -183,7 +183,7 @@
183 183 "shipped_days": 12
184 184 }
185 185 ],
186 − "expected": 2978.0
186 + "expected": 3031.0
187 187 },
188 188 {
189 189 "input": [
@@ -224,7 +224,7 @@
224 224 "shipped_days": 2
225 225 }
226 226 ],
227 − "expected": 5195.0
227 + "expected": 5404.0
228 228 },
229 229 {
230 230 "input": [
@@ -292,7 +292,7 @@
292 292 "shipped_days": 6
293 293 }
294 294 ],
295 − "expected": 6772.0
295 + "expected": 7044.0
296 296 },
297 297 {
298 298 "input": [
@@ -432,7 +432,7 @@
432 432 "shipped_days": 7
433 433 }
434 434 ],
435 − "expected": 11561.0
435 + "expected": 12047.0
436 436 }
437 437 ]
438 438 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00230_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00230_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "2e09e5c09991cb1b7c5c288b9a49bb058042a17dc7fd32cde98e2b705477598c",
5 + "document_hash": "cbb9dba4a75d2570390205536d928042eae0341c49e7dcc11a2cea988c175ba9",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9897: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 24 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -196,7 +196,7 @@
196 196 "shipped_days": 3
197 197 }
198 198 ],
199 − "expected": 9688.0
199 + "expected": 9995.0
200 200 },
201 201 {
202 202 "input": [
@@ -273,7 +273,7 @@
273 273 "shipped_days": 7
274 274 }
275 275 ],
276 − "expected": 3730.0
276 + "expected": 3834.0
277 277 },
278 278 {
279 279 "input": [
@@ -368,7 +368,7 @@
368 368 "shipped_days": 13
369 369 }
370 370 ],
371 − "expected": 10447.0
371 + "expected": 10628.0
372 372 },
373 373 {
374 374 "input": [
@@ -499,7 +499,7 @@
499 499 "shipped_days": 13
500 500 }
501 501 ],
502 − "expected": 7718.0
502 + "expected": 7815.0
503 503 },
504 504 {
505 505 "input": [
@@ -612,7 +612,7 @@
612 612 "shipped_days": 10
613 613 }
614 614 ],
615 − "expected": 7101.0
615 + "expected": 7378.0
616 616 }
617 617 ]
618 618 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00231_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00231_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "9071b939e284c02834a84c70b2af1783b2a26d6217aa2184b3224f0c27400eaf",
5 + "document_hash": "b599ec4f6ea744baef17b36d05ab828f105675eeebf81a780befd49f389ec9d3",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8917: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 754
30 + "char_end": 627
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -133,7 +133,7 @@
133 133 "shipped_days": 2
134 134 }
135 135 ],
136 − "expected": 4152.7
136 + "expected": 4283.8
137 137 },
138 138 {
139 139 "input": [
@@ -210,7 +210,7 @@
210 210 "shipped_days": 7
211 211 }
212 212 ],
213 − "expected": 4440.6
213 + "expected": 4715.0
214 214 },
215 215 {
216 216 "input": [
@@ -377,7 +377,7 @@
377 377 "shipped_days": 9
378 378 }
379 379 ],
380 − "expected": 9136.0
380 + "expected": 9560.7
381 381 },
382 382 {
383 383 "input": [
@@ -481,7 +481,7 @@
481 481 "shipped_days": 7
482 482 }
483 483 ],
484 − "expected": 5626.6
484 + "expected": 5933.4
485 485 },
486 486 {
487 487 "input": [
@@ -648,7 +648,7 @@
648 648 "shipped_days": 1
649 649 }
650 650 ],
651 − "expected": 9136.0
651 + "expected": 9560.7
652 652 }
653 653 ]
654 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00232_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00232_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "6bb55261f58b559a6304971baddf54cabc7de9843d30afe350a659e21fa02b73",
5 + "document_hash": "94e54bf28dac5a404fdf94451d3e632c18f63dff001cc758f89f5239d27f659b",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 3 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 3, \"min_units\": 5, \"excl_region\": \"Est\", \"rounding\": 2, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 3 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-8049: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 27 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -97,7 +97,7 @@
97 97 "shipped_days": 9
98 98 }
99 99 ],
100 − "expected": 2607.84
100 + "expected": 2758.36
101 101 },
102 102 {
103 103 "input": [
@@ -219,7 +219,7 @@
219 219 "shipped_days": 11
220 220 }
221 221 ],
222 − "expected": 5806.2
222 + "expected": 6158.4
223 223 },
224 224 {
225 225 "input": [
@@ -332,7 +332,7 @@
332 332 "shipped_days": 10
333 333 }
334 334 ],
335 − "expected": 5434.24
335 + "expected": 5765.79
336 336 },
337 337 {
338 338 "input": [
@@ -481,7 +481,7 @@
481 481 "shipped_days": 9
482 482 }
483 483 ],
484 − "expected": 5806.2
484 + "expected": 6158.4
485 485 },
486 486 {
487 487 "input": [
@@ -522,7 +522,7 @@
522 522 "shipped_days": 10
523 523 }
524 524 ],
525 − "expected": 2784.23
525 + "expected": 2949.0
526 526 }
527 527 ]
528 528 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00233_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00233_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "f88eb8610ee0ae9a0cc4c953e6f302bceb7e6a9f131c2e5f775a4026d048e997",
5 + "document_hash": "c0f56e7b9b73df623955c8b6822884e4b53df031c58736213f498ed2b4d693dc",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 1, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-6605: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 18 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 750
30 + "char_end": 623
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -160,7 +160,7 @@
160 160 "shipped_days": 6
161 161 }
162 162 ],
163 − "expected": 10002.2
163 + "expected": 10256.6
164 164 },
165 165 {
166 166 "input": [
@@ -273,7 +273,7 @@
273 273 "shipped_days": 3
274 274 }
275 275 ],
276 − "expected": 9483.9
276 + "expected": 9650.7
277 277 },
278 278 {
279 279 "input": [
@@ -436,7 +436,7 @@
436 436 "shipped_days": 9
437 437 }
438 438 ],
439 − "expected": 9655.3
439 + "expected": 9888.2
440 440 },
441 441 {
442 442 "input": [
@@ -513,7 +513,7 @@
513 513 "shipped_days": 6
514 514 }
515 515 ],
516 − "expected": 5501.4
516 + "expected": 5662.6
517 517 }
518 518 ]
519 519 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00234_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00234_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "89b49129e4581b4b424c4986c040ee62a62ff6f482d53151d96cbcf944c97e02",
5 + "document_hash": "3522724b66617b60e55fa67efc23998ff528130352e7a960ee1e052d697670a5",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 6 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Ouest':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 6, \"min_units\": 5, \"excl_region\": \"Ouest\", \"rounding\": 1, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 6 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-3367: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 28 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Ouest'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 753
30 + "char_end": 626
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -205,7 +205,7 @@
205 205 "shipped_days": 8
206 206 }
207 207 ],
208 − "expected": 6977.4
208 + "expected": 7260.4
209 209 },
210 210 {
211 211 "input": [
@@ -372,7 +372,7 @@
372 372 "shipped_days": 6
373 373 }
374 374 ],
375 − "expected": 6926.7
375 + "expected": 7206.5
376 376 },
377 377 {
378 378 "input": [
@@ -494,7 +494,7 @@
494 494 "shipped_days": 1
495 495 }
496 496 ],
497 − "expected": 4747.4
497 + "expected": 4987.8
498 498 },
499 499 {
500 500 "input": [
@@ -553,7 +553,7 @@
553 553 "shipped_days": 6
554 554 }
555 555 ],
556 − "expected": 3576.5
556 + "expected": 3677.4
557 557 },
558 558 {
559 559 "input": [
@@ -693,7 +693,7 @@
693 693 "shipped_days": 6
694 694 }
695 695 ],
696 − "expected": 5237.7
696 + "expected": 5505.7
697 697 }
698 698 ]
699 699 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00235_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00235_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "7870795b3e6c98bee59eb42d0e036ccc1d88fe696a6e7c94016c77fb0f3cefe4",
5 + "document_hash": "20bfca2f1ed7277720116b37988eb2bf9331505c61f7491ff7f131ed0b2d70e1",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 6:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 6, \"excl_region\": \"Sud\", \"rounding\": 0, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-2404: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 16 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 6 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 750
30 + "char_end": 623
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 106 "shipped_days": 7
107 107 }
108 108 ],
109 − "expected": 4951.0
109 + "expected": 5095.0
110 110 },
111 111 {
112 112 "input": [
@@ -174,7 +174,7 @@
174 174 "shipped_days": 2
175 175 }
176 176 ],
177 − "expected": 2359.0
177 + "expected": 2434.0
178 178 },
179 179 {
180 180 "input": [
@@ -278,7 +278,7 @@
278 278 "shipped_days": 2
279 279 }
280 280 ],
281 − "expected": 3596.0
281 + "expected": 3751.0
282 282 },
283 283 {
284 284 "input": [
@@ -337,7 +337,7 @@
337 337 "shipped_days": 1
338 338 }
339 339 ],
340 − "expected": 1146.0
340 + "expected": 1213.0
341 341 },
342 342 {
343 343 "input": [
@@ -441,7 +441,7 @@
441 441 "shipped_days": 11
442 442 }
443 443 ],
444 − "expected": 4940.0
444 + "expected": 5083.0
445 445 }
446 446 ]
447 447 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00236_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00236_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "dabe94079fad1aa1fef7653ff5583522be249aa0c33c58334ae1d261091d83f9",
5 + "document_hash": "23053422d45849043d9eb3706daf105a581cd66b69e96220ab1285c7c92b2058",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 7 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 9.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 7, \"min_units\": 3, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 9.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 7 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 9.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4238: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 25 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 9.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 757
30 + "char_end": 630
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 115 "shipped_days": 2
116 116 }
117 117 ],
118 − "expected": 5423.58
118 + "expected": 5614.82
119 119 },
120 120 {
121 121 "input": [
@@ -183,7 +183,7 @@
183 183 "shipped_days": 11
184 184 }
185 185 ],
186 − "expected": 4004.65
186 + "expected": 4149.91
187 187 },
188 188 {
189 189 "input": [
@@ -251,7 +251,7 @@
251 251 "shipped_days": 12
252 252 }
253 253 ],
254 − "expected": 3993.39
254 + "expected": 4134.77
255 255 },
256 256 {
257 257 "input": [
@@ -292,7 +292,7 @@
292 292 "shipped_days": 12
293 293 }
294 294 ],
295 − "expected": 2050.91
295 + "expected": 2142.17
296 296 },
297 297 {
298 298 "input": [
@@ -396,7 +396,7 @@
396 396 "shipped_days": 4
397 397 }
398 398 ],
399 − "expected": 5916.65
399 + "expected": 6153.7
400 400 }
401 401 ]
402 402 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00237_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00237_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "455d86e6a8f1e9be0eaebb2e2ab279c0439c6768273a410e7994bc6a76dfbecf",
5 + "document_hash": "a8320d4e9d153e4398af56ca92c73f39bb1c1b9e97609e9ec096c1157cebe872",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 9 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 9, \"min_units\": 5, \"excl_region\": \"Nord\", \"rounding\": 0, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 9 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-5627: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 11 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 755
30 + "char_end": 628
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -178,7 +178,7 @@
178 178 "shipped_days": 10
179 179 }
180 180 ],
181 − "expected": 11415.0
181 + "expected": 11767.0
182 182 },
183 183 {
184 184 "input": [
@@ -318,7 +318,7 @@
318 318 "shipped_days": 10
319 319 }
320 320 ],
321 − "expected": 12985.0
321 + "expected": 13403.0
322 322 },
323 323 {
324 324 "input": [
@@ -467,7 +467,7 @@
467 467 "shipped_days": 4
468 468 }
469 469 ],
470 − "expected": 12638.0
470 + "expected": 13134.0
471 471 },
472 472 {
473 473 "input": [
@@ -553,7 +553,7 @@
553 553 "shipped_days": 4
554 554 }
555 555 ],
556 − "expected": 5676.0
556 + "expected": 5875.0
557 557 },
558 558 {
559 559 "input": [
@@ -648,7 +648,7 @@
648 648 "shipped_days": 13
649 649 }
650 650 ],
651 − "expected": 7335.0
651 + "expected": 7713.0
652 652 }
653 653 ]
654 654 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00238_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00238_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "ffdd32b5c64c4cab201265cf785b962e0e60f2f65cf46cfd2674256d0a97fe14",
5 + "document_hash": "068a11a09e40cb07626e21d7abb095fb9c4779acc1a0db602ca10678ae79e3e0",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Est':\n continue\n if r['units'] < 4:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 5.0 / 100.0)\n return round(total, 0)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 4, \"excl_region\": \"Est\", \"rounding\": 0, \"tax\": 5.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 5.0 % on the sum; (7) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-3671: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 1 June 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Est'; (3) exclude rows with fewer than 4 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 5.0 % on the sum; (6) return the result rounded to 0 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 749
30 + "char_end": 622
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -88,7 +88,7 @@
88 88 "shipped_days": 3
89 89 }
90 90 ],
91 − "expected": 3129.0
91 + "expected": 3164.0
92 92 },
93 93 {
94 94 "input": [
@@ -210,7 +210,7 @@
210 210 "shipped_days": 14
211 211 }
212 212 ],
213 − "expected": 5186.0
213 + "expected": 5282.0
214 214 },
215 215 {
216 216 "input": [
@@ -260,7 +260,7 @@
260 260 "shipped_days": 13
261 261 }
262 262 ],
263 − "expected": 1745.0
263 + "expected": 1812.0
264 264 },
265 265 {
266 266 "input": [
@@ -445,7 +445,7 @@
445 445 "shipped_days": 1
446 446 }
447 447 ],
448 − "expected": 7412.0
448 + "expected": 7509.0
449 449 },
450 450 {
451 451 "input": [
@@ -594,7 +594,7 @@
594 594 "shipped_days": 11
595 595 }
596 596 ],
597 − "expected": 5050.0
597 + "expected": 5126.0
598 598 }
599 599 ]
600 600 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00239_2000.json +8 −8
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00239_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "1215eafbd18b1d396d2779e2c0227bfcff05e59f1f06d602fc8284f18da6bbc8",
5 + "document_hash": "015c78e1e480fb805976a26912f0308a79a0c4b2122313463b5ff6b5ee996a9b",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 4 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units'] < 5:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 14.975 / 100.0)\n return round(total, 2)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 4, \"min_units\": 5, \"excl_region\": \"Sud\", \"rounding\": 2, \"tax\": 14.975}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) rows shipped in more than 4 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 14.975 % on the sum; (7) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-4265: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 10 September 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Sud'; (3) exclude rows with fewer than 5 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 14.975 % on the sum; (6) return the result rounded to 2 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 758
30 + "char_end": 631
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -115,7 +115,7 @@
115 115 "shipped_days": 1
116 116 }
117 117 ],
118 − "expected": 3381.09
118 + "expected": 3407.42
119 119 },
120 120 {
121 121 "input": [
@@ -269,7 +269,7 @@
269 269 "shipped_days": 10
270 270 }
271 271 ],
272 − "expected": 3841.81
272 + "expected": 3877.51
273 273 },
274 274 {
275 275 "input": [
@@ -400,7 +400,7 @@
400 400 "shipped_days": 10
401 401 }
402 402 ],
403 − "expected": 8760.99
403 + "expected": 8961.66
404 404 },
405 405 {
406 406 "input": [
@@ -540,7 +540,7 @@
540 540 "shipped_days": 6
541 541 }
542 542 ],
543 − "expected": 8834.55
543 + "expected": 9035.23
544 544 }
545 545 ]
546 546 }
\ No newline at end of file
modified data/generated/keys/exec/exec_00240_2000.json +9 −9
@@ -2,13 +2,13 @@
2 2 "document_id": "exec_00240_2000",
3 3 "family": "exec",
4 4 "split": "test",
5 − "document_hash": "2598ae3b4041685f7fbf0de4eb4c12a5508a3c374ae89cdf7007d3cf7e6e5924",
5 + "document_hash": "4c3927b8378a2b949e85371101d959a0adebcbe41a2d39f239aae883de382e19",
6 6 "questions": [
7 7 {
8 8 "question_id": "q00_exec",
9 9 "qtype": "exec",
10 10 "question": "Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. Output only Python code defining net_revenue; no explanations, no markdown fences.",
11 − "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n d = r['discount_pct'] + (5 if r['shipped_days'] > 8 else 0)\n d = min(d, 50)\n total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
11 + "answer": "\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Nord':\n continue\n if r['units'] < 3:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total = total * (1 + 0.0 / 100.0)\n return round(total, 1)\n",
12 12 "answer_type": "exec",
13 13 "tolerance": 0.0,
14 14 "supporting_fact_ids": [
@@ -25,9 +25,9 @@
25 25 "attribute": "rules",
26 26 "value": "{\"thr_days\": 8, \"min_units\": 3, \"excl_region\": \"Nord\", \"rounding\": 1, \"tax\": 0.0}",
27 27 "kind": "spec",
28 − "sentence": "Bug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) rows shipped in more than 8 days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; (5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of 0.0 % on the sum; (7) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (4) and rule (2).",
28 + "sentence": "Bug report BR-9289: the function net_revenue(rows) must compute the total net revenue of the rows list (each row is a dict with keys ['region', 'product', 'units', 'unit_price', 'discount_pct', 'returned', 'shipped_days']). Rules agreed with the client on 2 March 2031: (1) exclude returned rows; (2) exclude rows whose region is 'Nord'; (3) exclude rows with fewer than 3 units; (4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of 0.0 % on the sum; (6) return the result rounded to 1 decimals (Python round). The current implementation forgets rule (2) and applies no tax.",
29 29 "char_start": 20,
30 − "char_end": 751
30 + "char_end": 624
31 31 }
32 32 ],
33 33 "hidden_tests": [
@@ -106,7 +106,7 @@
106 106 "shipped_days": 6
107 107 }
108 108 ],
109 − "expected": 4847.2
109 + "expected": 4959.7
110 110 },
111 111 {
112 112 "input": [
@@ -228,7 +228,7 @@
228 228 "shipped_days": 6
229 229 }
230 230 ],
231 − "expected": 9420.6
231 + "expected": 9646.5
232 232 },
233 233 {
234 234 "input": [
@@ -278,7 +278,7 @@
278 278 "shipped_days": 13
279 279 }
280 280 ],
281 − "expected": 3825.6
281 + "expected": 3866.6
282 282 },
283 283 {
284 284 "input": [
@@ -382,7 +382,7 @@
382 382 "shipped_days": 6
383 383 }
384 384 ],
385 − "expected": 7704.3
385 + "expected": 7897.0
386 386 },
387 387 {
388 388 "input": [
@@ -504,7 +504,7 @@
504 504 "shipped_days": 8
505 505 }
506 506 ],
507 − "expected": 9420.6
507 + "expected": 9646.5
508 508 }
509 509 ]
510 510 }
\ No newline at end of file
modified data/manifests/splits.json +181 −181
@@ -1,5 +1,5 @@
1 1 {
2 − "created_utc": "2026-10-30T03:19:45.688979Z",
2 + "created_utc": "2026-10-30T03:30:41.472673Z",
3 3 "config": {
4 4 "synthetic": {
5 5 "n_projects": 6,
@@ -1269,8 +1269,8 @@
1269 1269 "document_id": "exec_00001_2000",
1270 1270 "family": "exec",
1271 1271 "split": "train",
1272 − "document_hash": "ddfe7fcc3baca3e97144b66188f285a0e8a83018d1ca41b7f37cb80dede62dcc",
1273 − "char_len": 8073,
1272 + "document_hash": "72a702cc7076fb313f97772498675d83532bf6ce59f1d3cbf3bb86fa387dc0e6",
1273 + "char_len": 7894,
1274 1274 "n_questions": 1,
1275 1275 "cluster_id": "exec_00001_2000",
1276 1276 "seed": 1
@@ -1279,8 +1279,8 @@
1279 1279 "document_id": "exec_00002_2000",
1280 1280 "family": "exec",
1281 1281 "split": "train",
1282 − "document_hash": "7918e3cdcab2a8bfae77558b7b523f4cea70cdc52435393d98d0c7b36de29e60",
1283 − "char_len": 8001,
1282 + "document_hash": "7ec85eb17fda1c57c2a77a31dc79bf35b3c62974f6f1905e409880cbe869b6f5",
1283 + "char_len": 7819,
1284 1284 "n_questions": 1,
1285 1285 "cluster_id": "exec_00002_2000",
1286 1286 "seed": 2
@@ -1289,8 +1289,8 @@
1289 1289 "document_id": "exec_00003_2000",
1290 1290 "family": "exec",
1291 1291 "split": "train",
1292 − "document_hash": "8ab845867bddfdcb5cf63b1319a6f29794cc9dddea45cd92bca3bc94fe023da7",
1293 − "char_len": 7972,
1292 + "document_hash": "f7d729b32469d142625681408a1dd1aa3d1d99a7c07adb645c9f40d7550e8695",
1293 + "char_len": 7793,
1294 1294 "n_questions": 1,
1295 1295 "cluster_id": "exec_00003_2000",
1296 1296 "seed": 3
@@ -1299,8 +1299,8 @@
1299 1299 "document_id": "exec_00004_2000",
1300 1300 "family": "exec",
1301 1301 "split": "train",
1302 − "document_hash": "08c5cdfaa7a0742a094ee4726d97a32a89395c354270406bbcf78eff4fdbe8b2",
1303 − "char_len": 8037,
1302 + "document_hash": "aeca575de77be2cf25f1c9f8e053485ad918ef4b3468680a4ef6bfa3b8cb781b",
1303 + "char_len": 7855,
1304 1304 "n_questions": 1,
1305 1305 "cluster_id": "exec_00004_2000",
1306 1306 "seed": 4
@@ -1309,8 +1309,8 @@
1309 1309 "document_id": "exec_00005_2000",
1310 1310 "family": "exec",
1311 1311 "split": "train",
1312 − "document_hash": "f885d791d02ed1adb010c9c27804b5c38facf8fd2a09b9b1b6f9cacb7b0acc81",
1313 − "char_len": 8031,
1312 + "document_hash": "c5e26772985f1897f27e3729a56bdf95e08ba4718b928a1480572409c422c83f",
1313 + "char_len": 7852,
1314 1314 "n_questions": 1,
1315 1315 "cluster_id": "exec_00005_2000",
1316 1316 "seed": 5
@@ -1319,8 +1319,8 @@
1319 1319 "document_id": "exec_00006_2000",
1320 1320 "family": "exec",
1321 1321 "split": "train",
1322 − "document_hash": "2a1bb6d86c77861b166e281bb1deac9a658ed4383a7ade93e9698e33861de0e5",
1323 − "char_len": 8049,
1322 + "document_hash": "283461ef1dfaf2225620026d081a07116ec5fa2173ffa118704c0cf1d53a2eac",
1323 + "char_len": 7867,
1324 1324 "n_questions": 1,
1325 1325 "cluster_id": "exec_00006_2000",
1326 1326 "seed": 6
@@ -1329,8 +1329,8 @@
1329 1329 "document_id": "exec_00007_2000",
1330 1330 "family": "exec",
1331 1331 "split": "train",
1332 − "document_hash": "fb0150ce17a4b0ade1601cf6082b82e83f67db83a57c177064777d028b33d97c",
1333 − "char_len": 7826,
1332 + "document_hash": "961b89accd7b0332a165c291bb3c527fd00bea183807cff965326b5f758f1b7f",
1333 + "char_len": 7644,
1334 1334 "n_questions": 1,
1335 1335 "cluster_id": "exec_00007_2000",
1336 1336 "seed": 7
@@ -1339,8 +1339,8 @@
1339 1339 "document_id": "exec_00008_2000",
1340 1340 "family": "exec",
1341 1341 "split": "train",
1342 − "document_hash": "04f896fa681f3ae7cbfc7efa30fd96c43be9d9b703dab76457ee8043483b3612",
1343 − "char_len": 8052,
1342 + "document_hash": "4ad9a1fbe7db8f523841d367742505a5c92581010a15fdabb2bb6b6d77afbaa4",
1343 + "char_len": 7873,
1344 1344 "n_questions": 1,
1345 1345 "cluster_id": "exec_00008_2000",
1346 1346 "seed": 8
@@ -1349,8 +1349,8 @@
1349 1349 "document_id": "exec_00009_2000",
1350 1350 "family": "exec",
1351 1351 "split": "train",
1352 − "document_hash": "db44cbab6b33239dddaa205976960167be792c1b1025bd39a4c4d6cf0f6ab7d7",
1353 − "char_len": 8097,
1352 + "document_hash": "7c7609e3e3780713446d7ec4cbe48dba3ad69a1bf6d96393eb7e34fe13581d81",
1353 + "char_len": 7916,
1354 1354 "n_questions": 1,
1355 1355 "cluster_id": "exec_00009_2000",
1356 1356 "seed": 9
@@ -1359,8 +1359,8 @@
1359 1359 "document_id": "exec_00010_2000",
1360 1360 "family": "exec",
1361 1361 "split": "train",
1362 − "document_hash": "232fe52471a905a3ae14b591781c17dfa57a2d115258bfc1f8a1fce231b705ad",
1363 − "char_len": 8004,
1362 + "document_hash": "2951e556f1045edd4bb9926de1dac81793260ddcd823e8c6abe38566b4cbcc07",
1363 + "char_len": 7825,
1364 1364 "n_questions": 1,
1365 1365 "cluster_id": "exec_00010_2000",
1366 1366 "seed": 10
@@ -1369,8 +1369,8 @@
1369 1369 "document_id": "exec_00011_2000",
1370 1370 "family": "exec",
1371 1371 "split": "train",
1372 − "document_hash": "f046c8292307adf97cae067d41f5a7bcce9d2a48bc8dd143e0f11e89a1c3a4c2",
1373 − "char_len": 8135,
1372 + "document_hash": "6b5b870bbcdf744c667890cf6185717b10f87e45ad7a6ccada501987d53f5e6d",
1373 + "char_len": 7953,
1374 1374 "n_questions": 1,
1375 1375 "cluster_id": "exec_00011_2000",
1376 1376 "seed": 11
@@ -1379,8 +1379,8 @@
1379 1379 "document_id": "exec_00012_2000",
1380 1380 "family": "exec",
1381 1381 "split": "train",
1382 − "document_hash": "2befcba5bfd5b509a9a6ec5d9334e2e384a05b6733ac02d80222302d3ca6e6ef",
1383 − "char_len": 7906,
1382 + "document_hash": "8991dd153dfd83b4a1aea0eb4340aabe4370ac72ef16427f5f67894ac175edde",
1383 + "char_len": 7724,
1384 1384 "n_questions": 1,
1385 1385 "cluster_id": "exec_00012_2000",
1386 1386 "seed": 12
@@ -1389,8 +1389,8 @@
1389 1389 "document_id": "exec_00013_2000",
1390 1390 "family": "exec",
1391 1391 "split": "train",
1392 − "document_hash": "d006207e385cff8703be29071faf5845de84d05d8a677e40dda4983739115d29",
1393 − "char_len": 8051,
1392 + "document_hash": "ee93ee22294a4463cb6cf9ca3d5ab7ea7fd2a991b2b55448f089c2290e51a174",
1393 + "char_len": 7870,
1394 1394 "n_questions": 1,
1395 1395 "cluster_id": "exec_00013_2000",
1396 1396 "seed": 13
@@ -1399,8 +1399,8 @@
1399 1399 "document_id": "exec_00014_2000",
1400 1400 "family": "exec",
1401 1401 "split": "train",
1402 − "document_hash": "e6ac9234df3fc7dcec86770e041bf75b576d1c65cf6fe32717891429f5461066",
1403 − "char_len": 8089,
1402 + "document_hash": "b25f9d4fb9c997b0cb6e19983b48490097f6592b8d17e66f305f5a1b3e8a8443",
1403 + "char_len": 7910,
1404 1404 "n_questions": 1,
1405 1405 "cluster_id": "exec_00014_2000",
1406 1406 "seed": 14
@@ -1409,8 +1409,8 @@
1409 1409 "document_id": "exec_00015_2000",
1410 1410 "family": "exec",
1411 1411 "split": "train",
1412 − "document_hash": "7bd733e903db470812271aa82f1cd9c7111ace0dcd257ace35b07e660d045be7",
1413 − "char_len": 8041,
1412 + "document_hash": "7334bc5b260e11b2b0cdf9db05839cac2f397ae0e44db4e8bba7559dc94036bb",
1413 + "char_len": 7859,
1414 1414 "n_questions": 1,
1415 1415 "cluster_id": "exec_00015_2000",
1416 1416 "seed": 15
@@ -1419,8 +1419,8 @@
1419 1419 "document_id": "exec_00016_2000",
1420 1420 "family": "exec",
1421 1421 "split": "train",
1422 − "document_hash": "b232fee620440e55facf033b892741387c97661b269dc2371313fb0f5934ff69",
1423 − "char_len": 8045,
1422 + "document_hash": "6a5268852caa8dc3015fd42e678e912f463ba061ac417b09de9ac6ed45e85d7f",
1423 + "char_len": 7864,
1424 1424 "n_questions": 1,
1425 1425 "cluster_id": "exec_00016_2000",
1426 1426 "seed": 16
@@ -1429,8 +1429,8 @@
1429 1429 "document_id": "exec_00017_2000",
1430 1430 "family": "exec",
1431 1431 "split": "train",
1432 − "document_hash": "bce532bd3702b32e6eb3620ef3ea7d860a695f22bb90b8e995836a7937c77f52",
1433 − "char_len": 8053,
1432 + "document_hash": "f2d06422bd98b400b7401ce4c5ae2939aeb43f51482e84cba0c0e3c0845c71a5",
1433 + "char_len": 7872,
1434 1434 "n_questions": 1,
1435 1435 "cluster_id": "exec_00017_2000",
1436 1436 "seed": 17
@@ -1439,8 +1439,8 @@
1439 1439 "document_id": "exec_00018_2000",
1440 1440 "family": "exec",
1441 1441 "split": "train",
1442 − "document_hash": "6f0951d7af338593059efbe431f0b7ce5bf9ae652e3dfb9eedafa7e192b80c01",
1443 − "char_len": 8169,
1442 + "document_hash": "f219110fda4e5d20fabbe592248da3e78f61f411dbc16b9412d9b8d692eda870",
1443 + "char_len": 7990,
1444 1444 "n_questions": 1,
1445 1445 "cluster_id": "exec_00018_2000",
1446 1446 "seed": 18
@@ -1449,8 +1449,8 @@
1449 1449 "document_id": "exec_00019_2000",
1450 1450 "family": "exec",
1451 1451 "split": "train",
1452 − "document_hash": "733853d52a559dfc67d0cdab02de9e21c2aa09e0b25702d2cadfb38c20ab2159",
1453 − "char_len": 8031,
1452 + "document_hash": "232819b55ebc60215845a14aaf01c04a94dc7902933c03a47546ec41ee16502f",
1453 + "char_len": 7849,
1454 1454 "n_questions": 1,
1455 1455 "cluster_id": "exec_00019_2000",
1456 1456 "seed": 19
@@ -1459,8 +1459,8 @@
1459 1459 "document_id": "exec_00020_2000",
1460 1460 "family": "exec",
1461 1461 "split": "train",
1462 − "document_hash": "1a7bfffc3347254640c8b76b9b5f0ff368a758543995fa06006e28ea43d8eb71",
1463 − "char_len": 8080,
1462 + "document_hash": "862136f6ccb7f77097086ceddd84945e55b7df17401a63b0ce306740969966e5",
1463 + "char_len": 7901,
1464 1464 "n_questions": 1,
1465 1465 "cluster_id": "exec_00020_2000",
1466 1466 "seed": 20
@@ -1469,8 +1469,8 @@
1469 1469 "document_id": "exec_00101_2000",
1470 1470 "family": "exec",
1471 1471 "split": "dev",
1472 − "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9",
1473 − "char_len": 8125,
1472 + "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f",
1473 + "char_len": 7944,
1474 1474 "n_questions": 1,
1475 1475 "cluster_id": "exec_00101_2000",
1476 1476 "seed": 101
@@ -1479,8 +1479,8 @@
1479 1479 "document_id": "exec_00102_2000",
1480 1480 "family": "exec",
1481 1481 "split": "dev",
1482 − "document_hash": "5535ac3109616f7d46dff5417535ca232754aee2c0759e072d552e465f004d46",
1483 − "char_len": 8089,
1482 + "document_hash": "61b6c7bb86261a26bf000996edd5f9326f7f77ca1c37085fd999cddf1bff2396",
1483 + "char_len": 7910,
1484 1484 "n_questions": 1,
1485 1485 "cluster_id": "exec_00102_2000",
1486 1486 "seed": 102
@@ -1489,8 +1489,8 @@
1489 1489 "document_id": "exec_00103_2000",
1490 1490 "family": "exec",
1491 1491 "split": "dev",
1492 − "document_hash": "a7875d1e95742c6246c4ec3862b9797a4269a2b607e47869f5014d7dc29139e0",
1493 − "char_len": 8038,
1492 + "document_hash": "596ef1cd6d37fdac729520a738ed129e4ffaf619d372ad869d8188a0a5cac1ef",
1493 + "char_len": 7856,
1494 1494 "n_questions": 1,
1495 1495 "cluster_id": "exec_00103_2000",
1496 1496 "seed": 103
@@ -1499,8 +1499,8 @@
1499 1499 "document_id": "exec_00104_2000",
1500 1500 "family": "exec",
1501 1501 "split": "dev",
1502 − "document_hash": "bce3091895fc315bf981ac1272a7da1f00b0f64819f1ecfa2a6dd8a3d23f2207",
1503 − "char_len": 8014,
1502 + "document_hash": "6b4aadaf1b71672d6d8aed5a412c7aa490e3e0eb1c92e5df6db1cc9554f83483",
1503 + "char_len": 7835,
1504 1504 "n_questions": 1,
1505 1505 "cluster_id": "exec_00104_2000",
1506 1506 "seed": 104
@@ -1509,8 +1509,8 @@
1509 1509 "document_id": "exec_00105_2000",
1510 1510 "family": "exec",
1511 1511 "split": "dev",
1512 − "document_hash": "fe4b8857bf1774be1123bec85a5743da5c3551d53125ddf128efde401f3990c2",
1513 − "char_len": 8046,
1512 + "document_hash": "a59face90fe592d3d8a9b375aafefb775631dd798a60a424f981cceae1ec2996",
1513 + "char_len": 7867,
1514 1514 "n_questions": 1,
1515 1515 "cluster_id": "exec_00105_2000",
1516 1516 "seed": 105
@@ -1519,8 +1519,8 @@
1519 1519 "document_id": "exec_00106_2000",
1520 1520 "family": "exec",
1521 1521 "split": "dev",
1522 − "document_hash": "c3652bfa1632f0cc62d6512131ef519100143881225c8b4ee572c1876e7c6c7f",
1523 − "char_len": 7943,
1522 + "document_hash": "a78f55af374fd04eb589b193a9eca2f84cf04bcdb3fa60aaeb29fbe358c79464",
1523 + "char_len": 7761,
1524 1524 "n_questions": 1,
1525 1525 "cluster_id": "exec_00106_2000",
1526 1526 "seed": 106
@@ -1529,8 +1529,8 @@
1529 1529 "document_id": "exec_00107_2000",
1530 1530 "family": "exec",
1531 1531 "split": "dev",
1532 − "document_hash": "983dbcacdfae2acf0316690a7fe50cfa2523ae41beffd9de42356f2ebd3d1e79",
1533 − "char_len": 7930,
1532 + "document_hash": "83ea6e4f6c5d3a6eed7a921a930819d760ffb3dac7c22761dc9b4c28084904cc",
1533 + "char_len": 7751,
1534 1534 "n_questions": 1,
1535 1535 "cluster_id": "exec_00107_2000",
1536 1536 "seed": 107
@@ -1539,8 +1539,8 @@
1539 1539 "document_id": "exec_00108_2000",
1540 1540 "family": "exec",
1541 1541 "split": "dev",
1542 − "document_hash": "7ebb05d3714e237d074dfc9a69c5b83e99932f9041b5d4fe15c8fc7224ba7078",
1543 − "char_len": 8038,
1542 + "document_hash": "b9ccaaf72598ea6e12a10942f2697dd01ca891115b652043a2f6ce4cb68570cc",
1543 + "char_len": 7859,
1544 1544 "n_questions": 1,
1545 1545 "cluster_id": "exec_00108_2000",
1546 1546 "seed": 108
@@ -1549,8 +1549,8 @@
1549 1549 "document_id": "exec_00109_2000",
1550 1550 "family": "exec",
1551 1551 "split": "dev",
1552 − "document_hash": "680dab47e67683cb351941eea647197cb2b2ea472ff116582019ace0c543d461",
1553 − "char_len": 8073,
1552 + "document_hash": "d53ada6a135b4827dcacb2ebec142b5000ad21961fb0cd199fc7c609424d4d5a",
1553 + "char_len": 7894,
1554 1554 "n_questions": 1,
1555 1555 "cluster_id": "exec_00109_2000",
1556 1556 "seed": 109
@@ -1559,8 +1559,8 @@
1559 1559 "document_id": "exec_00110_2000",
1560 1560 "family": "exec",
1561 1561 "split": "dev",
1562 − "document_hash": "f1699da28903736ad58ec0640a277bb2eac030c80c49c22c027c273374df86fa",
1563 − "char_len": 8019,
1562 + "document_hash": "789f1f1a750d514137a988d21a0807cc30c8548da830f2eeef66854fc179fcf5",
1563 + "char_len": 7840,
1564 1564 "n_questions": 1,
1565 1565 "cluster_id": "exec_00110_2000",
1566 1566 "seed": 110
@@ -1569,8 +1569,8 @@
1569 1569 "document_id": "exec_00111_2000",
1570 1570 "family": "exec",
1571 1571 "split": "dev",
1572 − "document_hash": "e65e9534b60b15cdaadf465396c6385868f5a0ed4be5ba04c458937c2eff43ae",
1573 − "char_len": 7942,
1572 + "document_hash": "9f3d9d2caa1830b0bf0b983179f4de3251ec9bb8648c710bf43f0fb6c531ef85",
1573 + "char_len": 7763,
1574 1574 "n_questions": 1,
1575 1575 "cluster_id": "exec_00111_2000",
1576 1576 "seed": 111
@@ -1579,8 +1579,8 @@
1579 1579 "document_id": "exec_00112_2000",
1580 1580 "family": "exec",
1581 1581 "split": "dev",
1582 − "document_hash": "4a8db801f5e7f6b2108618514f90c37fbdbb61e902c4441c0d68548a5792da05",
1583 − "char_len": 7996,
1582 + "document_hash": "182cf1cbb5939b7e1ca9170096aa77a9cc3c2c4f8efb3195e8b4ad19b77a3f46",
1583 + "char_len": 7814,
1584 1584 "n_questions": 1,
1585 1585 "cluster_id": "exec_00112_2000",
1586 1586 "seed": 112
@@ -1589,8 +1589,8 @@
1589 1589 "document_id": "exec_00113_2000",
1590 1590 "family": "exec",
1591 1591 "split": "dev",
1592 − "document_hash": "7535b4d324a3e28b457e40fe41d18d52e75a9ed0271b411285e522c6c8c8200e",
1593 − "char_len": 8055,
1592 + "document_hash": "c89720bd112bca637c8fa3b4a98f7cc41a384be468eee47f5c44607ee6fc46f1",
1593 + "char_len": 7874,
1594 1594 "n_questions": 1,
1595 1595 "cluster_id": "exec_00113_2000",
1596 1596 "seed": 113
@@ -1599,8 +1599,8 @@
1599 1599 "document_id": "exec_00114_2000",
1600 1600 "family": "exec",
1601 1601 "split": "dev",
1602 − "document_hash": "e1faab8175075b8cf98d2e5e16181db3f5a8f103ed89d35efee7ba4684968699",
1603 − "char_len": 8042,
1602 + "document_hash": "86a377fe8f0559479ca21128972c50d14bd9e19d38d2bf49f763d402259c0aad",
1603 + "char_len": 7863,
1604 1604 "n_questions": 1,
1605 1605 "cluster_id": "exec_00114_2000",
1606 1606 "seed": 114
@@ -1609,8 +1609,8 @@
1609 1609 "document_id": "exec_00115_2000",
1610 1610 "family": "exec",
1611 1611 "split": "dev",
1612 − "document_hash": "4d6a9aa137c992493ef42f6758dbb35b873a4a392b00eae7152d6e45b68a3c1b",
1613 − "char_len": 8127,
1612 + "document_hash": "33ee1d20ea5713048e1335d3786ba0b10b3a5998a16200f09823d0c4cf0dcbe1",
1613 + "char_len": 7946,
1614 1614 "n_questions": 1,
1615 1615 "cluster_id": "exec_00115_2000",
1616 1616 "seed": 115
@@ -1619,8 +1619,8 @@
1619 1619 "document_id": "exec_00116_2000",
1620 1620 "family": "exec",
1621 1621 "split": "dev",
1622 − "document_hash": "2b6100ff4b04865ebf622b9749757f3b32215faa2e32f12d63be71b0c18fc7be",
1623 − "char_len": 7941,
1622 + "document_hash": "379dcf9e9ada65463c8d73e2cbb54ff33c8c46ea3154539a1d6df25bab2a0a0e",
1623 + "char_len": 7762,
1624 1624 "n_questions": 1,
1625 1625 "cluster_id": "exec_00116_2000",
1626 1626 "seed": 116
@@ -1629,8 +1629,8 @@
1629 1629 "document_id": "exec_00117_2000",
1630 1630 "family": "exec",
1631 1631 "split": "dev",
1632 − "document_hash": "6e53d886a35d8e51c98908f1e40e9314353c62a19e3197603509fd2a59c6bbf3",
1633 − "char_len": 8064,
1632 + "document_hash": "34a1cc9614d0c2d2f9766b2968ae3fa3eb2b6cc1a372750650ae7f7359325896",
1633 + "char_len": 7883,
1634 1634 "n_questions": 1,
1635 1635 "cluster_id": "exec_00117_2000",
1636 1636 "seed": 117
@@ -1639,8 +1639,8 @@
1639 1639 "document_id": "exec_00118_2000",
1640 1640 "family": "exec",
1641 1641 "split": "dev",
1642 − "document_hash": "5353584056feb451c43bd2199aa87e2f7064bbfb9485b6eb1f6a1387dddac0f6",
1643 − "char_len": 8050,
1642 + "document_hash": "58d2108f993d9baafb67222dc4bd97895639cb6c2410e675e3ed2c4233c8fee4",
1643 + "char_len": 7868,
1644 1644 "n_questions": 1,
1645 1645 "cluster_id": "exec_00118_2000",
1646 1646 "seed": 118
@@ -1649,8 +1649,8 @@
1649 1649 "document_id": "exec_00119_2000",
1650 1650 "family": "exec",
1651 1651 "split": "dev",
1652 − "document_hash": "f62c9b5ecd91caa0cf95af49bf6d40dd2bc3885255008434f482e56b7a2a2fae",
1653 − "char_len": 7901,
1652 + "document_hash": "98f121e7a33ff30acfcaf7a0346bbc04cc9f508e092e316490793765a8e0ab09",
1653 + "char_len": 7722,
1654 1654 "n_questions": 1,
1655 1655 "cluster_id": "exec_00119_2000",
1656 1656 "seed": 119
@@ -1659,8 +1659,8 @@
1659 1659 "document_id": "exec_00120_2000",
1660 1660 "family": "exec",
1661 1661 "split": "dev",
1662 − "document_hash": "5b1b1089a793b0953579b8ce004efe99282ffd8a6eb17b16c3ce50273c121aad",
1663 − "char_len": 7992,
1662 + "document_hash": "c5be33e5009123ba2c251884304b839ae6fc5163f0745c302cffbabf618b40cb",
1663 + "char_len": 7811,
1664 1664 "n_questions": 1,
1665 1665 "cluster_id": "exec_00120_2000",
1666 1666 "seed": 120
@@ -1669,8 +1669,8 @@
1669 1669 "document_id": "exec_00121_2000",
1670 1670 "family": "exec",
1671 1671 "split": "dev",
1672 − "document_hash": "9064e893e331010c27db6e1229c2af7da83ed7fb3680e079fc5c1a9ee7b2d73a",
1673 − "char_len": 8033,
1672 + "document_hash": "1645df3901b5bd61ea8c6790e15a695b1144eef69bc075121ffd0c83e6fc13af",
1673 + "char_len": 7854,
1674 1674 "n_questions": 1,
1675 1675 "cluster_id": "exec_00121_2000",
1676 1676 "seed": 121
@@ -1679,8 +1679,8 @@
1679 1679 "document_id": "exec_00122_2000",
1680 1680 "family": "exec",
1681 1681 "split": "dev",
1682 − "document_hash": "8641f3a09b033f2a216948ea197682c280cda7a9559b7102473da5f152a73eb3",
1683 − "char_len": 8139,
1682 + "document_hash": "bd26e436a86876d04961cc142588671db4ead753f838933bde6907da5354f70e",
1683 + "char_len": 7960,
1684 1684 "n_questions": 1,
1685 1685 "cluster_id": "exec_00122_2000",
1686 1686 "seed": 122
@@ -1689,8 +1689,8 @@
1689 1689 "document_id": "exec_00123_2000",
1690 1690 "family": "exec",
1691 1691 "split": "dev",
1692 − "document_hash": "3a8a85801b1200bb0a18063e20eb67097e3e9c34d009894a5a61963f7145608b",
1693 − "char_len": 8069,
1692 + "document_hash": "6c868c5916de1b984d70dc327ee458814726f8ea6a1fad99f8c2fb35e7e77b7f",
1693 + "char_len": 7890,
1694 1694 "n_questions": 1,
1695 1695 "cluster_id": "exec_00123_2000",
1696 1696 "seed": 123
@@ -1699,8 +1699,8 @@
1699 1699 "document_id": "exec_00124_2000",
1700 1700 "family": "exec",
1701 1701 "split": "dev",
1702 − "document_hash": "adfa94aa242540ae900f58c2db73dace6e0bb6ec243a71137685b2d527e54fdb",
1703 − "char_len": 7919,
1702 + "document_hash": "a285cd3f7f5bbfcaa508697bf261341dc847483a97e354b95faf09bc23bb626f",
1703 + "char_len": 7740,
1704 1704 "n_questions": 1,
1705 1705 "cluster_id": "exec_00124_2000",
1706 1706 "seed": 124
@@ -1709,8 +1709,8 @@
1709 1709 "document_id": "exec_00125_2000",
1710 1710 "family": "exec",
1711 1711 "split": "dev",
1712 − "document_hash": "144a4af055228fc4045ad15b1ca9ba8229c8e5563cc4c4cf9ad94203a0072dd9",
1713 − "char_len": 7915,
1712 + "document_hash": "386fcaff7daca426267d33af6f30657f21e4863b66d5a71d38d0b78a334eb069",
1713 + "char_len": 7733,
1714 1714 "n_questions": 1,
1715 1715 "cluster_id": "exec_00125_2000",
1716 1716 "seed": 125
@@ -1719,8 +1719,8 @@
1719 1719 "document_id": "exec_00126_2000",
1720 1720 "family": "exec",
1721 1721 "split": "dev",
1722 − "document_hash": "a6bed1ca1c6bee001ded092712010d4dc8d3ec10d0d1f85432e8844be28b9359",
1723 − "char_len": 7944,
1722 + "document_hash": "0729452f4b76415c15bdd0075e98c4db16464515a675ea3d8e3611adb3c8a3e9",
1723 + "char_len": 7763,
1724 1724 "n_questions": 1,
1725 1725 "cluster_id": "exec_00126_2000",
1726 1726 "seed": 126
@@ -1729,8 +1729,8 @@
1729 1729 "document_id": "exec_00127_2000",
1730 1730 "family": "exec",
1731 1731 "split": "dev",
1732 − "document_hash": "01e9b8b715cbfdd0d66c26d571044c18f9a6d9d2c5146b834291716879cf02d2",
1733 − "char_len": 7953,
1732 + "document_hash": "bb6e6467042a1b7809f3ea38b83ea82e5554ca36b70042023f682efd5bae752a",
1733 + "char_len": 7774,
1734 1734 "n_questions": 1,
1735 1735 "cluster_id": "exec_00127_2000",
1736 1736 "seed": 127
@@ -1739,8 +1739,8 @@
1739 1739 "document_id": "exec_00128_2000",
1740 1740 "family": "exec",
1741 1741 "split": "dev",
1742 − "document_hash": "e702731f232f4b91d0513add4b301fd1ef0382eee35df22a98cef7eb727fafca",
1743 − "char_len": 7945,
1742 + "document_hash": "ea7fe68afa60ebc43cd0a501c243c72cd3b993bfaef6e389f12308e1967e3471",
1743 + "char_len": 7763,
1744 1744 "n_questions": 1,
1745 1745 "cluster_id": "exec_00128_2000",
1746 1746 "seed": 128
@@ -1749,8 +1749,8 @@
1749 1749 "document_id": "exec_00129_2000",
1750 1750 "family": "exec",
1751 1751 "split": "dev",
1752 − "document_hash": "e508132470ec83c8745c1899d6c954b865130cad33382718fa48d35d6bea0ec8",
1753 − "char_len": 7973,
1752 + "document_hash": "46f3b799f18b08ee443481845e8695eff10ac2f2b23c2b177eafce3b42e2ea40",
1753 + "char_len": 7794,
1754 1754 "n_questions": 1,
1755 1755 "cluster_id": "exec_00129_2000",
1756 1756 "seed": 129
@@ -1759,8 +1759,8 @@
1759 1759 "document_id": "exec_00130_2000",
1760 1760 "family": "exec",
1761 1761 "split": "dev",
1762 − "document_hash": "c65f14fbcb1d4d0049a6e641be2826817637866851032f95c4cc1943f87ff809",
1763 − "char_len": 8072,
1762 + "document_hash": "c415053d25119b89bd769d02be02c390efebcd68b992b946ddf043ad29041f86",
1763 + "char_len": 7893,
1764 1764 "n_questions": 1,
1765 1765 "cluster_id": "exec_00130_2000",
1766 1766 "seed": 130
@@ -1769,8 +1769,8 @@
1769 1769 "document_id": "exec_00201_2000",
1770 1770 "family": "exec",
1771 1771 "split": "test",
1772 − "document_hash": "876f090ca1801217ec4295964e7d9ce5b46b9fc96b04db7ca463d6cc8ed74978",
1773 − "char_len": 8073,
1772 + "document_hash": "065b6c18bbd16f16126924a54946ca4bf95c5863e57b14f33e4c9d88568f8549",
1773 + "char_len": 7892,
1774 1774 "n_questions": 1,
1775 1775 "cluster_id": "exec_00201_2000",
1776 1776 "seed": 201
@@ -1779,8 +1779,8 @@
1779 1779 "document_id": "exec_00202_2000",
1780 1780 "family": "exec",
1781 1781 "split": "test",
1782 − "document_hash": "660fc77600d7958889cef9b34f9c9a1a8c2d96ef38db054d180f225ebe3d4a42",
1783 − "char_len": 7844,
1782 + "document_hash": "ec13194237af7efe7765a21f1d8303ab8ac449271eb48e3395906648c770bab8",
1783 + "char_len": 7665,
1784 1784 "n_questions": 1,
1785 1785 "cluster_id": "exec_00202_2000",
1786 1786 "seed": 202
@@ -1789,8 +1789,8 @@
1789 1789 "document_id": "exec_00203_2000",
1790 1790 "family": "exec",
1791 1791 "split": "test",
1792 − "document_hash": "f26e049133c3f806d2045fa255ba4e6c2594f5ecbe872414b75faaf52ceb5033",
1793 − "char_len": 7967,
1792 + "document_hash": "39eee37c7ba5fe1b6d8fce9eef4c00e10a33d8ca049d12ae01b9521d4ac60863",
1793 + "char_len": 7788,
1794 1794 "n_questions": 1,
1795 1795 "cluster_id": "exec_00203_2000",
1796 1796 "seed": 203
@@ -1799,8 +1799,8 @@
1799 1799 "document_id": "exec_00204_2000",
1800 1800 "family": "exec",
1801 1801 "split": "test",
1802 − "document_hash": "7cc296695d97fdf8fb07de5772744af9c3a79a52f20ee1e3410187cac4dd94b2",
1803 − "char_len": 8100,
1802 + "document_hash": "0aaf1a095160aa21c4485db3b1b171e86d9d41b904ac2a2ea102cc13df19cd22",
1803 + "char_len": 7921,
1804 1804 "n_questions": 1,
1805 1805 "cluster_id": "exec_00204_2000",
1806 1806 "seed": 204
@@ -1809,8 +1809,8 @@
1809 1809 "document_id": "exec_00205_2000",
1810 1810 "family": "exec",
1811 1811 "split": "test",
1812 − "document_hash": "1f9b534774ca095126415c70bc29f4400d55f083ec020a658e6ee1263a76072f",
1813 − "char_len": 8061,
1812 + "document_hash": "c82f149ddbd77be80773d103825621fb187f06bb60d4fd570c73c2b3cd898ecb",
1813 + "char_len": 7882,
1814 1814 "n_questions": 1,
1815 1815 "cluster_id": "exec_00205_2000",
1816 1816 "seed": 205
@@ -1819,8 +1819,8 @@
1819 1819 "document_id": "exec_00206_2000",
1820 1820 "family": "exec",
1821 1821 "split": "test",
1822 − "document_hash": "71e5bf98eb246c53babd99cdd8ec1212e594a1af3ddd2a6da69bac721ad65ddd",
1823 − "char_len": 7908,
1822 + "document_hash": "e391c9d757bc03677c664222c3ff3f1bbab881d7f566c0072b184e0b90503339",
1823 + "char_len": 7727,
1824 1824 "n_questions": 1,
1825 1825 "cluster_id": "exec_00206_2000",
1826 1826 "seed": 206
@@ -1829,8 +1829,8 @@
1829 1829 "document_id": "exec_00207_2000",
1830 1830 "family": "exec",
1831 1831 "split": "test",
1832 − "document_hash": "2273befa734c54f0d780db1126ec3ce69ce8f9f829e57586a614786258681ba3",
1833 − "char_len": 8035,
1832 + "document_hash": "7d024469b5d38776a899aa870f5af7e146953a19f1adc1d91f634cda62aa4c14",
1833 + "char_len": 7856,
1834 1834 "n_questions": 1,
1835 1835 "cluster_id": "exec_00207_2000",
1836 1836 "seed": 207
@@ -1839,8 +1839,8 @@
1839 1839 "document_id": "exec_00208_2000",
1840 1840 "family": "exec",
1841 1841 "split": "test",
1842 − "document_hash": "e22e990c3c20d7feee2f70e993961e465ac68d3a867c3547dc0566739847c2d8",
1843 − "char_len": 8036,
1842 + "document_hash": "8528b3fafcf58b9daa88ffa9d7b9b25fb6e225f9bb767849744c05510a5ace37",
1843 + "char_len": 7854,
1844 1844 "n_questions": 1,
1845 1845 "cluster_id": "exec_00208_2000",
1846 1846 "seed": 208
@@ -1849,8 +1849,8 @@
1849 1849 "document_id": "exec_00209_2000",
1850 1850 "family": "exec",
1851 1851 "split": "test",
1852 − "document_hash": "bd895760d266f605848302775beaceb129633108ddc74dcbe83a8642eecb0ea3",
1853 − "char_len": 8044,
1852 + "document_hash": "27a0f4409b3611173adf72865b52c6beef014bce0282fd0498be50deeed192c9",
1853 + "char_len": 7865,
1854 1854 "n_questions": 1,
1855 1855 "cluster_id": "exec_00209_2000",
1856 1856 "seed": 209
@@ -1859,8 +1859,8 @@
1859 1859 "document_id": "exec_00210_2000",
1860 1860 "family": "exec",
1861 1861 "split": "test",
1862 − "document_hash": "2678fb8184c3c22ce8bfa861af1a68ef2704dd84cb88718d63c1b668adaf1f69",
1863 − "char_len": 7990,
1862 + "document_hash": "ffa82307f9b4986f3fc473b8b7df1e4f8f0e1c25d3b6e36cdcf4114a763427c5",
1863 + "char_len": 7811,
1864 1864 "n_questions": 1,
1865 1865 "cluster_id": "exec_00210_2000",
1866 1866 "seed": 210
@@ -1869,8 +1869,8 @@
1869 1869 "document_id": "exec_00211_2000",
1870 1870 "family": "exec",
1871 1871 "split": "test",
1872 − "document_hash": "5ad4fd06e4149025eddafab6fbf5c707e06c1f7e47f3cbaec839f52cd6744e38",
1873 − "char_len": 8091,
1872 + "document_hash": "95faa51b0302db54273e9b4469b6eae92c251463238fdad494d437435c293548",
1873 + "char_len": 7909,
1874 1874 "n_questions": 1,
1875 1875 "cluster_id": "exec_00211_2000",
1876 1876 "seed": 211
@@ -1879,8 +1879,8 @@
1879 1879 "document_id": "exec_00212_2000",
1880 1880 "family": "exec",
1881 1881 "split": "test",
1882 − "document_hash": "8a171bf76e4dc9c8921e9b2a4370fc5385ceb9695d62d19fc693110c84df06f0",
1883 − "char_len": 8061,
1882 + "document_hash": "2be5dc8496484a99fc11a00be474f4f7f139a8f491a2ebf3fff0db8e7804dd82",
1883 + "char_len": 7882,
1884 1884 "n_questions": 1,
1885 1885 "cluster_id": "exec_00212_2000",
1886 1886 "seed": 212
@@ -1889,8 +1889,8 @@
1889 1889 "document_id": "exec_00213_2000",
1890 1890 "family": "exec",
1891 1891 "split": "test",
1892 − "document_hash": "55fc33fe81ccfe25e3ebdcdb70ee135b990fcf94d02bfb56e0116982ffd6df6f",
1893 − "char_len": 7876,
1892 + "document_hash": "3c65c42333c25f1f3f90331747f08fafce263c8faa11a56d3a68bc58b2217327",
1893 + "char_len": 7695,
1894 1894 "n_questions": 1,
1895 1895 "cluster_id": "exec_00213_2000",
1896 1896 "seed": 213
@@ -1899,8 +1899,8 @@
1899 1899 "document_id": "exec_00214_2000",
1900 1900 "family": "exec",
1901 1901 "split": "test",
1902 − "document_hash": "0be81f6d80f5eb03ba862cbc94c8bf20b6ab1196483c947401585e7178fd458d",
1903 − "char_len": 7997,
1902 + "document_hash": "ac0a3ee69ec7da626616f6ffb5e443a4f26a3d7857caaa20f1d4ebfa5355d28b",
1903 + "char_len": 7815,
1904 1904 "n_questions": 1,
1905 1905 "cluster_id": "exec_00214_2000",
1906 1906 "seed": 214
@@ -1909,8 +1909,8 @@
1909 1909 "document_id": "exec_00215_2000",
1910 1910 "family": "exec",
1911 1911 "split": "test",
1912 − "document_hash": "37b95a047621c2a78beaff095b27647064c994bf6ede6ef6b3e91f8296b70eff",
1913 − "char_len": 8073,
1912 + "document_hash": "a4094c70181359b8442ead384be9497e3fb0c46a4d9266cce07ca77bd9e26e2b",
1913 + "char_len": 7894,
1914 1914 "n_questions": 1,
1915 1915 "cluster_id": "exec_00215_2000",
1916 1916 "seed": 215
@@ -1919,8 +1919,8 @@
1919 1919 "document_id": "exec_00216_2000",
1920 1920 "family": "exec",
1921 1921 "split": "test",
1922 − "document_hash": "1deddf301b0b2291cb9bdc4213848eceb17ece12032a2e454bcceb4ca4174397",
1923 − "char_len": 8017,
1922 + "document_hash": "1a218de32e6fe4bbe97927ea16330bda3b598ed66c1517824d94ff90f944e802",
1923 + "char_len": 7838,
1924 1924 "n_questions": 1,
1925 1925 "cluster_id": "exec_00216_2000",
1926 1926 "seed": 216
@@ -1929,8 +1929,8 @@
1929 1929 "document_id": "exec_00217_2000",
1930 1930 "family": "exec",
1931 1931 "split": "test",
1932 − "document_hash": "7f71cb12d994e6fe2646a0f7b337c441f923868ec6b21c29533122fda4d87a73",
1933 − "char_len": 7946,
1932 + "document_hash": "1223c7391b1da1aeae2d242c0e41de4639218cfe0c90ba9065f1cce42f0eedbd",
1933 + "char_len": 7764,
1934 1934 "n_questions": 1,
1935 1935 "cluster_id": "exec_00217_2000",
1936 1936 "seed": 217
@@ -1939,8 +1939,8 @@
1939 1939 "document_id": "exec_00218_2000",
1940 1940 "family": "exec",
1941 1941 "split": "test",
1942 − "document_hash": "2a77fd38142317b7e8868dfcaebb42248ddd462e9339268378a26e4b587eab3d",
1943 − "char_len": 7955,
1942 + "document_hash": "971d94c4e7b4ae02142ffb6eb92af9d2cc0f6e4d52e94a64c5132b510197ce8a",
1943 + "char_len": 7776,
1944 1944 "n_questions": 1,
1945 1945 "cluster_id": "exec_00218_2000",
1946 1946 "seed": 218
@@ -1949,8 +1949,8 @@
1949 1949 "document_id": "exec_00219_2000",
1950 1950 "family": "exec",
1951 1951 "split": "test",
1952 − "document_hash": "501d502ac1cd6920ddaeccbc4f0c925ddf4ca966cfb601ffd9d0547d97a741ed",
1953 − "char_len": 8119,
1952 + "document_hash": "390edd848add1a4cf011660e8ac9a6a6aa381c4508a8f68635f3217ce20979cb",
1953 + "char_len": 7938,
1954 1954 "n_questions": 1,
1955 1955 "cluster_id": "exec_00219_2000",
1956 1956 "seed": 219
@@ -1959,8 +1959,8 @@
1959 1959 "document_id": "exec_00220_2000",
1960 1960 "family": "exec",
1961 1961 "split": "test",
1962 − "document_hash": "a7487c2eb12039fcc44c64460641febdd19eaf0e13d8b6b8f00d018ca564a8b6",
1963 − "char_len": 7931,
1962 + "document_hash": "31eca9e7afc70ac5a795edc1d1529c531e0d028342802a61f4496c013e13bf45",
1963 + "char_len": 7750,
1964 1964 "n_questions": 1,
1965 1965 "cluster_id": "exec_00220_2000",
1966 1966 "seed": 220
@@ -1969,8 +1969,8 @@
1969 1969 "document_id": "exec_00221_2000",
1970 1970 "family": "exec",
1971 1971 "split": "test",
1972 − "document_hash": "a986eec8e76aa658fd694272657b8e201d940be0671e7769a934ffe76ca4748a",
1973 − "char_len": 7820,
1972 + "document_hash": "e228dfaa0d04173ee28b43513e5fc06914edf5d0c02b09bf0fc534531a583669",
1973 + "char_len": 7638,
1974 1974 "n_questions": 1,
1975 1975 "cluster_id": "exec_00221_2000",
1976 1976 "seed": 221
@@ -1979,8 +1979,8 @@
1979 1979 "document_id": "exec_00222_2000",
1980 1980 "family": "exec",
1981 1981 "split": "test",
1982 − "document_hash": "fa31c738b8b80170524adedf048965eb739a7ebf7de9fcd4f4192d331dcc3346",
1983 − "char_len": 8057,
1982 + "document_hash": "040260f25a42680fcc038386368506adaa004bc2592e75e3fbb5903b54cc19ab",
1983 + "char_len": 7878,
1984 1984 "n_questions": 1,
1985 1985 "cluster_id": "exec_00222_2000",
1986 1986 "seed": 222
@@ -1989,8 +1989,8 @@
1989 1989 "document_id": "exec_00223_2000",
1990 1990 "family": "exec",
1991 1991 "split": "test",
1992 − "document_hash": "9b2d718a88e7b451a6e6d433e81fa6c3862777588f40791fe888b2d7a44efa42",
1993 − "char_len": 7992,
1992 + "document_hash": "7eef13486249d89cf05bda367ea9dae2ce2479b2236c1917b2f067c49a981619",
1993 + "char_len": 7813,
1994 1994 "n_questions": 1,
1995 1995 "cluster_id": "exec_00223_2000",
1996 1996 "seed": 223
@@ -1999,8 +1999,8 @@
1999 1999 "document_id": "exec_00224_2000",
2000 2000 "family": "exec",
2001 2001 "split": "test",
2002 − "document_hash": "526a53c484a99dde8d83b8fc83d103060128d6331659920d7688b8836de1ec53",
2003 − "char_len": 7921,
2002 + "document_hash": "3e642abb57841365b827053f2437a250dcb419539389820bac778d4596c064fd",
2003 + "char_len": 7739,
2004 2004 "n_questions": 1,
2005 2005 "cluster_id": "exec_00224_2000",
2006 2006 "seed": 224
@@ -2009,8 +2009,8 @@
2009 2009 "document_id": "exec_00225_2000",
2010 2010 "family": "exec",
2011 2011 "split": "test",
2012 − "document_hash": "0d5d08bcef45a939605c15cb0d0b5b1ac3c2304c8358dae05fa7da75db07621f",
2013 − "char_len": 7998,
2012 + "document_hash": "54918c16a4ae3706a87d182112fca2a39c82c77e3a60fd25a8b47b958f2a6b21",
2013 + "char_len": 7819,
2014 2014 "n_questions": 1,
2015 2015 "cluster_id": "exec_00225_2000",
2016 2016 "seed": 225
@@ -2019,8 +2019,8 @@
2019 2019 "document_id": "exec_00226_2000",
2020 2020 "family": "exec",
2021 2021 "split": "test",
2022 − "document_hash": "32ca9297d301e1956917680ad370eda2add0304f9742c49a2023cfa3f2a82bd2",
2023 − "char_len": 7972,
2022 + "document_hash": "7a73067df93b17aba1a608bce48f7a6ff16d480af8d23713ec889405f279fc7d",
2023 + "char_len": 7793,
2024 2024 "n_questions": 1,
2025 2025 "cluster_id": "exec_00226_2000",
2026 2026 "seed": 226
@@ -2029,8 +2029,8 @@
2029 2029 "document_id": "exec_00227_2000",
2030 2030 "family": "exec",
2031 2031 "split": "test",
2032 − "document_hash": "0297d0253d8ecb7ba91156d4221aff1d54ba5efaed7fe7664734e63d44d31bd0",
2033 − "char_len": 8074,
2032 + "document_hash": "6afe19e10d57be0082db797187045b004680fa22a13a56a7b8f7e2a51b40597d",
2033 + "char_len": 7895,
2034 2034 "n_questions": 1,
2035 2035 "cluster_id": "exec_00227_2000",
2036 2036 "seed": 227
@@ -2039,8 +2039,8 @@
2039 2039 "document_id": "exec_00228_2000",
2040 2040 "family": "exec",
2041 2041 "split": "test",
2042 − "document_hash": "7bd8eb7ac7409f89d5f44f1f5001fd0660aece03d12d92588e821c86e0d1f69b",
2043 − "char_len": 7994,
2042 + "document_hash": "08d5c40fa24cccbcccaac14d408c241ab602604a712b727f12f570eb56887a74",
2043 + "char_len": 7815,
2044 2044 "n_questions": 1,
2045 2045 "cluster_id": "exec_00228_2000",
2046 2046 "seed": 228
@@ -2049,8 +2049,8 @@
2049 2049 "document_id": "exec_00229_2000",
2050 2050 "family": "exec",
2051 2051 "split": "test",
2052 − "document_hash": "488c5efd60f8e64b88269226071b8befe058a5e2442b99934157b38f5fc41a19",
2053 − "char_len": 7905,
2052 + "document_hash": "b6a34443a51bb95703036fc2467d0c50ad248f99f3c11c0a35d6bc7fcf0ad481",
2053 + "char_len": 7723,
2054 2054 "n_questions": 1,
2055 2055 "cluster_id": "exec_00229_2000",
2056 2056 "seed": 229
@@ -2059,8 +2059,8 @@
2059 2059 "document_id": "exec_00230_2000",
2060 2060 "family": "exec",
2061 2061 "split": "test",
2062 − "document_hash": "2e09e5c09991cb1b7c5c288b9a49bb058042a17dc7fd32cde98e2b705477598c",
2063 − "char_len": 8012,
2062 + "document_hash": "cbb9dba4a75d2570390205536d928042eae0341c49e7dcc11a2cea988c175ba9",
2063 + "char_len": 7830,
2064 2064 "n_questions": 1,
2065 2065 "cluster_id": "exec_00230_2000",
2066 2066 "seed": 230
@@ -2069,8 +2069,8 @@
2069 2069 "document_id": "exec_00231_2000",
2070 2070 "family": "exec",
2071 2071 "split": "test",
2072 − "document_hash": "9071b939e284c02834a84c70b2af1783b2a26d6217aa2184b3224f0c27400eaf",
2073 − "char_len": 8077,
2072 + "document_hash": "b599ec4f6ea744baef17b36d05ab828f105675eeebf81a780befd49f389ec9d3",
2073 + "char_len": 7895,
2074 2074 "n_questions": 1,
2075 2075 "cluster_id": "exec_00231_2000",
2076 2076 "seed": 231
@@ -2079,8 +2079,8 @@
2079 2079 "document_id": "exec_00232_2000",
2080 2080 "family": "exec",
2081 2081 "split": "test",
2082 − "document_hash": "6bb55261f58b559a6304971baddf54cabc7de9843d30afe350a659e21fa02b73",
2083 − "char_len": 7990,
2082 + "document_hash": "94e54bf28dac5a404fdf94451d3e632c18f63dff001cc758f89f5239d27f659b",
2083 + "char_len": 7811,
2084 2084 "n_questions": 1,
2085 2085 "cluster_id": "exec_00232_2000",
2086 2086 "seed": 232
@@ -2089,8 +2089,8 @@
2089 2089 "document_id": "exec_00233_2000",
2090 2090 "family": "exec",
2091 2091 "split": "test",
2092 − "document_hash": "f88eb8610ee0ae9a0cc4c953e6f302bceb7e6a9f131c2e5f775a4026d048e997",
2093 − "char_len": 7909,
2092 + "document_hash": "c0f56e7b9b73df623955c8b6822884e4b53df031c58736213f498ed2b4d693dc",
2093 + "char_len": 7730,
2094 2094 "n_questions": 1,
2095 2095 "cluster_id": "exec_00233_2000",
2096 2096 "seed": 233
@@ -2099,8 +2099,8 @@
2099 2099 "document_id": "exec_00234_2000",
2100 2100 "family": "exec",
2101 2101 "split": "test",
2102 − "document_hash": "89b49129e4581b4b424c4986c040ee62a62ff6f482d53151d96cbcf944c97e02",
2103 − "char_len": 8006,
2102 + "document_hash": "3522724b66617b60e55fa67efc23998ff528130352e7a960ee1e052d697670a5",
2103 + "char_len": 7827,
2104 2104 "n_questions": 1,
2105 2105 "cluster_id": "exec_00234_2000",
2106 2106 "seed": 234
@@ -2109,8 +2109,8 @@
2109 2109 "document_id": "exec_00235_2000",
2110 2110 "family": "exec",
2111 2111 "split": "test",
2112 − "document_hash": "7870795b3e6c98bee59eb42d0e036ccc1d88fe696a6e7c94016c77fb0f3cefe4",
2113 − "char_len": 8079,
2112 + "document_hash": "20bfca2f1ed7277720116b37988eb2bf9331505c61f7491ff7f131ed0b2d70e1",
2113 + "char_len": 7900,
2114 2114 "n_questions": 1,
2115 2115 "cluster_id": "exec_00235_2000",
2116 2116 "seed": 235
@@ -2119,8 +2119,8 @@
2119 2119 "document_id": "exec_00236_2000",
2120 2120 "family": "exec",
2121 2121 "split": "test",
2122 − "document_hash": "dabe94079fad1aa1fef7653ff5583522be249aa0c33c58334ae1d261091d83f9",
2123 − "char_len": 8048,
2122 + "document_hash": "23053422d45849043d9eb3706daf105a581cd66b69e96220ab1285c7c92b2058",
2123 + "char_len": 7867,
2124 2124 "n_questions": 1,
2125 2125 "cluster_id": "exec_00236_2000",
2126 2126 "seed": 236
@@ -2129,8 +2129,8 @@
2129 2129 "document_id": "exec_00237_2000",
2130 2130 "family": "exec",
2131 2131 "split": "test",
2132 − "document_hash": "455d86e6a8f1e9be0eaebb2e2ab279c0439c6768273a410e7994bc6a76dfbecf",
2133 − "char_len": 8024,
2132 + "document_hash": "a8320d4e9d153e4398af56ca92c73f39bb1c1b9e97609e9ec096c1157cebe872",
2133 + "char_len": 7842,
2134 2134 "n_questions": 1,
2135 2135 "cluster_id": "exec_00237_2000",
2136 2136 "seed": 237
@@ -2139,8 +2139,8 @@
2139 2139 "document_id": "exec_00238_2000",
2140 2140 "family": "exec",
2141 2141 "split": "test",
2142 − "document_hash": "ffdd32b5c64c4cab201265cf785b962e0e60f2f65cf46cfd2674256d0a97fe14",
2143 − "char_len": 7855,
2142 + "document_hash": "068a11a09e40cb07626e21d7abb095fb9c4779acc1a0db602ca10678ae79e3e0",
2143 + "char_len": 7676,
2144 2144 "n_questions": 1,
2145 2145 "cluster_id": "exec_00238_2000",
2146 2146 "seed": 238
@@ -2149,8 +2149,8 @@
2149 2149 "document_id": "exec_00239_2000",
2150 2150 "family": "exec",
2151 2151 "split": "test",
2152 − "document_hash": "1215eafbd18b1d396d2779e2c0227bfcff05e59f1f06d602fc8284f18da6bbc8",
2153 − "char_len": 7905,
2152 + "document_hash": "015c78e1e480fb805976a26912f0308a79a0c4b2122313463b5ff6b5ee996a9b",
2153 + "char_len": 7723,
2154 2154 "n_questions": 1,
2155 2155 "cluster_id": "exec_00239_2000",
2156 2156 "seed": 239
@@ -2159,8 +2159,8 @@
2159 2159 "document_id": "exec_00240_2000",
2160 2160 "family": "exec",
2161 2161 "split": "test",
2162 − "document_hash": "2598ae3b4041685f7fbf0de4eb4c12a5508a3c374ae89cdf7007d3cf7e6e5924",
2163 − "char_len": 7976,
2162 + "document_hash": "4c3927b8378a2b949e85371101d959a0adebcbe41a2d39f239aae883de382e19",
2163 + "char_len": 7797,
2164 2164 "n_questions": 1,
2165 2165 "cluster_id": "exec_00240_2000",
2166 2166 "seed": 240
added protocol/pilot_matrix.csv +1489 −0
@@ -0,0 +1,1489 @@
1 +experiment_id,condition,method,access,sender,receiver,family,document_id,split,target_tokens,n_questions,seed,expected_trials
2 +E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
3 +E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
4 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
5 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
6 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
7 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
8 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
9 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
10 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
11 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
12 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
13 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
14 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
15 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
16 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
17 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
18 +E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
19 +E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
20 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
21 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
22 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
23 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
24 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
25 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
26 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
27 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
28 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
29 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
30 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
31 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
32 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02000_2000,dev,2000,8,0,8
33 +E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
34 +E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
35 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
36 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
37 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
38 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
39 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
40 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
41 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
42 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
43 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
44 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
45 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
46 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
47 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
48 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
49 +E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
50 +E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
51 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
52 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
53 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
54 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
55 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
56 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
57 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
58 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
59 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
60 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
61 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
62 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
63 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02000_2000,dev,2000,8,0,8
64 +E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
65 +E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
66 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
67 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
68 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
69 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
70 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
71 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
72 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
73 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
74 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
75 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
76 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
77 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
78 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
79 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
80 +E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
81 +E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
82 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
83 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
84 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
85 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
86 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
87 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
88 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
89 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
90 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
91 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
92 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
93 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
94 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02001_8000,dev,8000,8,0,8
95 +E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
96 +E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
97 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
98 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
99 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
100 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
101 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
102 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
103 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
104 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
105 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
106 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
107 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
108 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
109 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
110 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
111 +E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
112 +E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
113 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
114 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
115 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
116 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
117 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
118 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
119 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
120 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
121 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
122 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
123 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
124 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
125 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02001_8000,dev,8000,8,0,8
126 +E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
127 +E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
128 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
129 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
130 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
131 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
132 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
133 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
134 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
135 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
136 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
137 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
138 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
139 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
140 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
141 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
142 +E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
143 +E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
144 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
145 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
146 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
147 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
148 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
149 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
150 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
151 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
152 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
153 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
154 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
155 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
156 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02003_2000,dev,2000,8,0,8
157 +E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
158 +E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
159 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
160 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
161 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
162 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
163 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
164 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
165 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
166 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
167 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
168 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
169 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
170 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
171 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
172 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
173 +E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
174 +E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
175 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
176 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
177 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
178 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
179 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
180 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
181 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
182 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
183 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
184 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
185 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
186 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
187 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02003_2000,dev,2000,8,0,8
188 +E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
189 +E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
190 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
191 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
192 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
193 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
194 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
195 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
196 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
197 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
198 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
199 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
200 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
201 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
202 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
203 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
204 +E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
205 +E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
206 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
207 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
208 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
209 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
210 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
211 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
212 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
213 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
214 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
215 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
216 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
217 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
218 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02004_8000,dev,8000,8,0,8
219 +E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
220 +E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
221 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
222 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
223 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
224 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
225 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
226 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
227 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
228 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
229 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
230 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
231 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
232 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
233 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
234 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
235 +E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
236 +E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
237 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
238 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
239 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
240 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
241 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
242 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
243 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
244 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
245 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
246 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
247 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
248 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
249 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02004_8000,dev,8000,8,0,8
250 +E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
251 +E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
252 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
253 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
254 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
255 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
256 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
257 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
258 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
259 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
260 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
261 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
262 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
263 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
264 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
265 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
266 +E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
267 +E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
268 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
269 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
270 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
271 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
272 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
273 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
274 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
275 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
276 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
277 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
278 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
279 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
280 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02006_2000,dev,2000,8,0,8
281 +E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
282 +E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
283 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
284 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
285 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
286 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
287 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
288 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
289 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
290 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
291 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
292 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
293 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
294 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
295 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
296 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
297 +E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
298 +E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
299 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
300 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
301 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
302 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
303 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
304 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
305 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
306 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
307 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
308 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
309 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
310 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
311 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02006_2000,dev,2000,8,0,8
312 +E1_pilot,A0,A0,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
313 +E1_pilot,A1,A1,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
314 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
315 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
316 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
317 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
318 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
319 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
320 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
321 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
322 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
323 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
324 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
325 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
326 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
327 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
328 +E1_pilot,B0,B0,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
329 +E1_pilot,B1,B1,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
330 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
331 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
332 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
333 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
334 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
335 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
336 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
337 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
338 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
339 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
340 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
341 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
342 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,synthetic,syn_02007_8000,dev,8000,8,0,8
343 +E1_pilot,A0,A0,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
344 +E1_pilot,A1,A1,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
345 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
346 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
347 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
348 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
349 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
350 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
351 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
352 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
353 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
354 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
355 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
356 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
357 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
358 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
359 +E1_pilot,B0,B0,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
360 +E1_pilot,B1,B1,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
361 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
362 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
363 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
364 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
365 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
366 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
367 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
368 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
369 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
370 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
371 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
372 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
373 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,synthetic,syn_02007_8000,dev,8000,8,0,8
374 +E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
375 +E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
376 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
377 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
378 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
379 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
380 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
381 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
382 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
383 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
384 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
385 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
386 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
387 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
388 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
389 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
390 +E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
391 +E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
392 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
393 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
394 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
395 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
396 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
397 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
398 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
399 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
400 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
401 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
402 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
403 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
404 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00101_2000,dev,,1,0,1
405 +E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
406 +E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
407 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
408 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
409 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
410 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
411 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
412 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
413 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
414 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
415 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
416 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
417 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
418 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
419 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
420 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
421 +E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
422 +E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
423 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
424 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
425 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
426 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
427 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
428 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
429 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
430 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
431 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
432 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
433 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
434 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
435 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00101_2000,dev,,1,0,1
436 +E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
437 +E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
438 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
439 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
440 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
441 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
442 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
443 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
444 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
445 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
446 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
447 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
448 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
449 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
450 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
451 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
452 +E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
453 +E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
454 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
455 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
456 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
457 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
458 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
459 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
460 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
461 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
462 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
463 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
464 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
465 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
466 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00102_2000,dev,,1,0,1
467 +E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
468 +E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
469 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
470 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
471 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
472 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
473 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
474 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
475 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
476 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
477 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
478 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
479 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
480 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
481 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
482 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
483 +E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
484 +E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
485 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
486 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
487 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
488 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
489 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
490 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
491 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
492 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
493 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
494 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
495 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
496 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
497 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00102_2000,dev,,1,0,1
498 +E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
499 +E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
500 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
501 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
502 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
503 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
504 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
505 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
506 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
507 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
508 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
509 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
510 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
511 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
512 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
513 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
514 +E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
515 +E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
516 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
517 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
518 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
519 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
520 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
521 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
522 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
523 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
524 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
525 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
526 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
527 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
528 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00103_2000,dev,,1,0,1
529 +E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
530 +E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
531 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
532 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
533 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
534 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
535 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
536 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
537 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
538 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
539 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
540 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
541 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
542 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
543 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
544 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
545 +E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
546 +E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
547 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
548 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
549 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
550 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
551 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
552 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
553 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
554 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
555 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
556 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
557 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
558 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
559 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00103_2000,dev,,1,0,1
560 +E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
561 +E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
562 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
563 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
564 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
565 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
566 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
567 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
568 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
569 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
570 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
571 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
572 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
573 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
574 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
575 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
576 +E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
577 +E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
578 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
579 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
580 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
581 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
582 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
583 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
584 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
585 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
586 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
587 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
588 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
589 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
590 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00104_2000,dev,,1,0,1
591 +E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
592 +E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
593 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
594 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
595 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
596 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
597 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
598 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
599 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
600 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
601 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
602 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
603 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
604 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
605 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
606 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
607 +E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
608 +E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
609 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
610 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
611 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
612 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
613 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
614 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
615 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
616 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
617 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
618 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
619 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
620 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
621 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00104_2000,dev,,1,0,1
622 +E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
623 +E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
624 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
625 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
626 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
627 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
628 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
629 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
630 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
631 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
632 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
633 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
634 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
635 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
636 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
637 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
638 +E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
639 +E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
640 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
641 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
642 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
643 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
644 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
645 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
646 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
647 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
648 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
649 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
650 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
651 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
652 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00105_2000,dev,,1,0,1
653 +E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
654 +E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
655 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
656 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
657 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
658 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
659 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
660 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
661 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
662 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
663 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
664 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
665 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
666 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
667 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
668 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
669 +E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
670 +E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
671 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
672 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
673 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
674 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
675 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
676 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
677 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
678 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
679 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
680 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
681 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
682 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
683 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00105_2000,dev,,1,0,1
684 +E1_pilot,A0,A0,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
685 +E1_pilot,A1,A1,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
686 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
687 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
688 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
689 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
690 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
691 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
692 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
693 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
694 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
695 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
696 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
697 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
698 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
699 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
700 +E1_pilot,B0,B0,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
701 +E1_pilot,B1,B1,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
702 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
703 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
704 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
705 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
706 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
707 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
708 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
709 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
710 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
711 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
712 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
713 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
714 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,exec,exec_00106_2000,dev,,1,0,1
715 +E1_pilot,A0,A0,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
716 +E1_pilot,A1,A1,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
717 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
718 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
719 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
720 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
721 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
722 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
723 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
724 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
725 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
726 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
727 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
728 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
729 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
730 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
731 +E1_pilot,B0,B0,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
732 +E1_pilot,B1,B1,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
733 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
734 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
735 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
736 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
737 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
738 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
739 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
740 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
741 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
742 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
743 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
744 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
745 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,exec,exec_00106_2000,dev,,1,0,1
746 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
747 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
748 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
749 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
750 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
751 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
752 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
753 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
754 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
755 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
756 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
757 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
758 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
759 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
760 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
761 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
762 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
763 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
764 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
765 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
766 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
767 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
768 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
769 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
770 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
771 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
772 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
773 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
774 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
775 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
776 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
777 +E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
778 +E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
779 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
780 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
781 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
782 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
783 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
784 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
785 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
786 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
787 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
788 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
789 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
790 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
791 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
792 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
793 +E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
794 +E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
795 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
796 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
797 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
798 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
799 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
800 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
801 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
802 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
803 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
804 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
805 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
806 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
807 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084,dev,,1,0,1
808 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
809 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
810 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
811 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
812 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
813 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
814 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
815 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
816 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
817 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
818 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
819 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
820 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
821 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
822 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
823 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
824 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
825 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
826 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
827 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
828 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
829 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
830 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
831 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
832 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
833 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
834 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
835 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
836 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
837 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
838 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
839 +E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
840 +E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
841 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
842 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
843 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
844 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
845 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
846 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
847 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
848 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
849 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
850 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
851 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
852 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
853 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
854 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
855 +E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
856 +E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
857 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
858 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
859 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
860 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
861 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
862 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
863 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
864 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
865 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
866 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
867 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
868 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
869 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_e419f2bff9d2ab7c3b60b3250caccd2d9ae1285ec3e8e818,dev,,1,0,1
870 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
871 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
872 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
873 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
874 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
875 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
876 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
877 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
878 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
879 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
880 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
881 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
882 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
883 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
884 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
885 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
886 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
887 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
888 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
889 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
890 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
891 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
892 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
893 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
894 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
895 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
896 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
897 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
898 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
899 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
900 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
901 +E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
902 +E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
903 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
904 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
905 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
906 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
907 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
908 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
909 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
910 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
911 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
912 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
913 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
914 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
915 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
916 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
917 +E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
918 +E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
919 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
920 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
921 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
922 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
923 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
924 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
925 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
926 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
927 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
928 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
929 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
930 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
931 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_9fb085a1f47673d1907f2378c90843b4b6e8622a14fe1fa9,dev,,1,0,1
932 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
933 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
934 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
935 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
936 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
937 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
938 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
939 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
940 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
941 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
942 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
943 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
944 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
945 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
946 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
947 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
948 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
949 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
950 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
951 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
952 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
953 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
954 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
955 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
956 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
957 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
958 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
959 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
960 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
961 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
962 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
963 +E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
964 +E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
965 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
966 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
967 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
968 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
969 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
970 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
971 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
972 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
973 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
974 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
975 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
976 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
977 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
978 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
979 +E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
980 +E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
981 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
982 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
983 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
984 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
985 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
986 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
987 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
988 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
989 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
990 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
991 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
992 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
993 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_c109a25d23a8c5c95af8697ef1ac51cd3bee93c67521af6d,dev,,1,0,1
994 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
995 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
996 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
997 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
998 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
999 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1000 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1001 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1002 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1003 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1004 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1005 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1006 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1007 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1008 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1009 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1010 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1011 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1012 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1013 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1014 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1015 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1016 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1017 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1018 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1019 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1020 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1021 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1022 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1023 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1024 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1025 +E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1026 +E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1027 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1028 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1029 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1030 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1031 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1032 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1033 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1034 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1035 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1036 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1037 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1038 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1039 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1040 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1041 +E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1042 +E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1043 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1044 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1045 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1046 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1047 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1048 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1049 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1050 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1051 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1052 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1053 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1054 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1055 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_3b069a546d5417be546f57d02981cf5ac68b4d3d6c55624a,dev,,1,0,1
1056 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1057 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1058 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1059 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1060 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1061 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1062 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1063 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1064 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1065 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1066 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1067 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1068 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1069 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1070 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1071 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1072 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1073 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1074 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1075 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1076 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1077 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1078 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1079 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1080 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1081 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1082 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1083 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1084 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1085 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1086 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1087 +E1_pilot,A0,A0,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1088 +E1_pilot,A1,A1,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1089 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1090 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1091 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1092 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1093 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1094 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1095 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1096 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1097 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1098 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1099 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1100 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1101 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1102 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1103 +E1_pilot,B0,B0,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1104 +E1_pilot,B1,B1,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1105 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1106 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1107 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1108 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1109 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1110 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1111 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1112 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1113 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1114 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1115 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1116 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1117 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_qasper,lb_qasper_091b4028a3b5e9d8248c58f17a62fd16c878da69693cfbfb,dev,,1,0,1
1118 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1119 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1120 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1121 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1122 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1123 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1124 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1125 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1126 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1127 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1128 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1129 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1130 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1131 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1132 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1133 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1134 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1135 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1136 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1137 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1138 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1139 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1140 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1141 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1142 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1143 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1144 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1145 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1146 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1147 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1148 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1149 +E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1150 +E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1151 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1152 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1153 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1154 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1155 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1156 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1157 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1158 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1159 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1160 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1161 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1162 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1163 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1164 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1165 +E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1166 +E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1167 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1168 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1169 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1170 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1171 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1172 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1173 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1174 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1175 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1176 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1177 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1178 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1179 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_3924e4ac5039ce3fadda49604bfcb0f5238af81774616e53,dev,,1,0,1
1180 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1181 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1182 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1183 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1184 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1185 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1186 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1187 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1188 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1189 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1190 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1191 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1192 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1193 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1194 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1195 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1196 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1197 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1198 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1199 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1200 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1201 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1202 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1203 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1204 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1205 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1206 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1207 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1208 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1209 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1210 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1211 +E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1212 +E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1213 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1214 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1215 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1216 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1217 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1218 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1219 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1220 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1221 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1222 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1223 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1224 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1225 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1226 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1227 +E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1228 +E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1229 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1230 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1231 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1232 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1233 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1234 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1235 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1236 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1237 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1238 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1239 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1240 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1241 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4b28d517ce1c1e3cfec9282ca7b212c1cb87c254781d7c86,dev,,1,0,1
1242 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1243 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1244 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1245 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1246 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1247 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1248 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1249 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1250 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1251 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1252 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1253 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1254 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1255 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1256 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1257 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1258 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1259 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1260 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1261 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1262 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1263 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1264 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1265 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1266 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1267 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1268 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1269 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1270 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1271 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1272 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1273 +E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1274 +E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1275 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1276 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1277 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1278 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1279 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1280 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1281 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1282 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1283 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1284 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1285 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1286 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1287 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1288 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1289 +E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1290 +E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1291 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1292 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1293 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1294 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1295 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1296 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1297 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1298 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1299 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1300 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1301 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1302 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1303 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_048e82b64b5651b74d452db7151c2110a718128dfd12a774,dev,,1,0,1
1304 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1305 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1306 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1307 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1308 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1309 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1310 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1311 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1312 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1313 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1314 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1315 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1316 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1317 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1318 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1319 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1320 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1321 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1322 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1323 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1324 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1325 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1326 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1327 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1328 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1329 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1330 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1331 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1332 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1333 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1334 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1335 +E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1336 +E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1337 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1338 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1339 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1340 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1341 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1342 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1343 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1344 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1345 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1346 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1347 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1348 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1349 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1350 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1351 +E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1352 +E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1353 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1354 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1355 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1356 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1357 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1358 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1359 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1360 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1361 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1362 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1363 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1364 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1365 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4db5bcd1d49fce674507d9128850eb71b808b7dc4246e882,dev,,1,0,1
1366 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1367 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1368 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1369 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1370 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1371 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1372 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1373 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1374 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1375 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1376 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1377 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1378 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1379 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1380 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1381 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1382 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1383 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1384 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1385 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1386 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1387 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1388 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1389 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1390 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1391 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1392 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1393 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1394 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1395 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1396 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1397 +E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1398 +E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1399 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1400 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1401 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1402 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1403 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1404 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1405 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1406 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1407 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1408 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1409 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1410 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1411 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1412 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1413 +E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1414 +E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1415 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1416 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1417 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1418 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1419 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1420 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1421 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1422 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1423 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1424 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1425 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1426 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1427 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_2a398d925d3607c94ffb2d0cf9fe2fe6da6e9970ce95578a,dev,,1,0,1
1428 +E1_pilot,A0,A0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1429 +E1_pilot,A1,A1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1430 +E1_pilot,A2|budget_tokens=128,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1431 +E1_pilot,A2|budget_tokens=512,A2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1432 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1433 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1434 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1435 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1436 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1437 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1438 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1439 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1440 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1441 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1442 +E1_pilot,A7|codec=zstd,A7,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1443 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1444 +E1_pilot,B0,B0,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1445 +E1_pilot,B1,B1,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1446 +E1_pilot,B2|bits=8,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1447 +E1_pilot,B2|bits=4,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1448 +E1_pilot,B2|bits=2|group_size=32,B2,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1449 +E1_pilot,B3|keep_frac=0.5,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1450 +E1_pilot,B3|keep_frac=0.25,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1451 +E1_pilot,B3|keep_frac=0.125,B3,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1452 +E1_pilot,B4|keep_frac=0.25,B4,none,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1453 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1454 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1455 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1456 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1457 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1458 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,qwen3b,qwen3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1459 +E1_pilot,A0,A0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1460 +E1_pilot,A1,A1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1461 +E1_pilot,A2|budget_tokens=128,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1462 +E1_pilot,A2|budget_tokens=512,A2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1463 +E1_pilot,A3|budget_tokens=128|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1464 +E1_pilot,A3|budget_tokens=512|blind,A3,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1465 +E1_pilot,A3|budget_tokens=512|aware,A3,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1466 +E1_pilot,A4|budget_tokens=128|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1467 +E1_pilot,A4|budget_tokens=512|blind,A4,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1468 +E1_pilot,A4|budget_tokens=512|aware,A4,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1469 +E1_pilot,A5|budget_tokens=128|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1470 +E1_pilot,A5|budget_tokens=512|blind,A5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1471 +E1_pilot,A5|budget_tokens=512|aware,A5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1472 +E1_pilot,A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind,A6,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1473 +E1_pilot,A7|codec=zstd,A7,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1474 +E1_pilot,A8|budget_tokens=512|blind,A8,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1475 +E1_pilot,B0,B0,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1476 +E1_pilot,B1,B1,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1477 +E1_pilot,B2|bits=8,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1478 +E1_pilot,B2|bits=4,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1479 +E1_pilot,B2|bits=2|group_size=32,B2,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1480 +E1_pilot,B3|keep_frac=0.5,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1481 +E1_pilot,B3|keep_frac=0.25,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1482 +E1_pilot,B3|keep_frac=0.125,B3,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1483 +E1_pilot,B4|keep_frac=0.25,B4,none,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1484 +E1_pilot,B5|keep_frac=0.5|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1485 +E1_pilot,B5|keep_frac=0.25|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1486 +E1_pilot,B5|keep_frac=0.125|aware,B5,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1487 +E1_pilot,B5|keep_frac=0.25|blind,B5,blind,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1488 +E1_pilot,B6|keep_frac=0.25|aware,B6,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
1489 +E1_pilot,B9|bits=8|keep_frac=0.25|policy=attention|aware,B9,aware,llama3b,llama3b,public_2wikimqa,lb_2wikimqa_4413705182b57f12dc553e0919b50fd3c3b111cb4d82d12f,dev,,1,0,1
added runs/raw/E0_smoke.jsonl +114 −0
@@ -0,0 +1,114 @@
1 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4146ad264bc41966c53c6c2a", "attempt_id": "82062e437bf8", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:34.723314Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 64, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 704.172583995387, "ttft_ms": 111.37787497136742, "decode_ms": 52.97070904634893}, "total_wall_ms": 874.542957986705, "ttft_ms": 111.37787497136742, "input_usage": {"receiver_prompt_tokens": 64}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6598574080, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6221848952, "mlx_cache_bytes": 52763976, "process_rss_bytes": 6598574080}, "peak_prefill_memory": null, "peak_decode_memory": 6221848952, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
2 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "aea3215c06f1768bed89e0e4", "attempt_id": "eb2cf16a0cb8", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:35.621639Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 70, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 714.0282499603927, "ttft_ms": 16.376250074245036, "decode_ms": 20.608124905265868}, "total_wall_ms": 751.6555410111323, "ttft_ms": 16.376250074245036, "input_usage": {"receiver_prompt_tokens": 70}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6599262208, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6243951240, "mlx_cache_bytes": 117497160, "process_rss_bytes": 6599262208}, "peak_prefill_memory": null, "peak_decode_memory": 6243951240, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
3 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "19cb26d87f4f6c06f22133ab", "attempt_id": "f2ace8e95ee4", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:36.394852Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 70, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 37.55691600963473, "ttft_ms": 13.19108298048377, "decode_ms": 21.748042083345354}, "total_wall_ms": 73.1698750751093, "ttft_ms": 13.19108298048377, "input_usage": {"receiver_prompt_tokens": 70}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6599311360, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6243951240, "mlx_cache_bytes": 117497160, "process_rss_bytes": 6599311360}, "peak_prefill_memory": null, "peak_decode_memory": 6243951240, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
4 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "55040e08a970feabf0c8dfb0", "attempt_id": "d1f4af490c6e", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:36.488122Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 67, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.007208087481558323, "decode_payload_ms": 0.0, "receiver_prefill_ms": 33.89208402950317, "ttft_ms": 13.485999894328415, "decode_ms": 13.414166984148324}, "total_wall_ms": 61.261874972842634, "ttft_ms": 13.485999894328415, "input_usage": {"receiver_prompt_tokens": 67}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6599344128, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6245671560, "mlx_cache_bytes": 162520392, "process_rss_bytes": 6599344128}, "peak_prefill_memory": null, "peak_decode_memory": 6245671560, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
5 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b5a907e579d3ba6be2a73c6e", "attempt_id": "8a37232cf18e", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:36.569588Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2045, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 268.84245907422155, "ttft_ms": 40.27962498366833, "decode_ms": 220.9290829487145}, "total_wall_ms": 533.2459580386057, "ttft_ms": 40.27962498366833, "input_usage": {"receiver_prompt_tokens": 2045}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6599606272, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7061651504, "mlx_cache_bytes": 1144192312, "process_rss_bytes": 6599606272}, "peak_prefill_memory": null, "peak_decode_memory": 7061651504, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
6 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c3b51441b577e5cf867a711f", "attempt_id": "899cf42e0e05", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:37.123667Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 660.5961669702083, "ttft_ms": 19.695666036568582, "decode_ms": 177.584792021662}, "total_wall_ms": 861.3416670123115, "ttft_ms": 19.695666036568582, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600179712, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 1950271808, "process_rss_bytes": 6600179712}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
7 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "768e16d34f259a76a877fe66", "attempt_id": "56658b192495", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:38.004317Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 270.62474994454533, "ttft_ms": 13.784875045530498, "decode_ms": 137.02083297539502}, "total_wall_ms": 424.99795800540596, "ttft_ms": 13.784875045530498, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6600196096, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 1950271808, "process_rss_bytes": 6600196096}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
8 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bf2332e5ea92995e70a698fa", "attempt_id": "b65f009d2226", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:38.450459Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2048, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013124896213412285, "decode_payload_ms": 0.0, "receiver_prefill_ms": 270.55058302357793, "ttft_ms": 16.825958038680255, "decode_ms": 115.82745797932148}, "total_wall_ms": 406.5157499862835, "ttft_ms": 16.825958038680255, "input_usage": {"receiver_prompt_tokens": 2048}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6600228864, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7063175232, "mlx_cache_bytes": 1950271808, "process_rss_bytes": 6600228864}, "peak_prefill_memory": null, "peak_decode_memory": 7063175232, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
9 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "680a1696a501a9abbd246fe8", "attempt_id": "7116fac29eef", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:38.879959Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 595, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 98.40858390089124, "ttft_ms": 14.796082978136837, "decode_ms": 28.082999982871115}, "total_wall_ms": 142.88441697135568, "ttft_ms": 14.796082978136837, "input_usage": {"receiver_prompt_tokens": 595}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600474624, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6566965336, "mlx_cache_bytes": 2345519424, "process_rss_bytes": 6600474624}, "peak_prefill_memory": null, "peak_decode_memory": 6566965336, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
10 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "7d73c49fb6ea7ba015627e4f", "attempt_id": "c1a192f5a6d1", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.046782Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 601, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 96.09833301510662, "ttft_ms": 21.742249955423176, "decode_ms": 187.9519580397755}, "total_wall_ms": 307.20599996857345, "ttft_ms": 21.742249955423176, "input_usage": {"receiver_prompt_tokens": 601}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600540160, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 2695874880, "process_rss_bytes": 6600540160}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C in Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "3491362e6ed6bb557ea2c25e29286d6fdbd0a0a36995d38446eddfd8450e097c"}
11 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "610ae196b469cb2b3d8c27c1", "attempt_id": "ee38ce288b68", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.376514Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 601, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 105.36962503101677, "ttft_ms": 16.225583967752755, "decode_ms": 25.455374969169497}, "total_wall_ms": 148.63470802083611, "ttft_ms": 16.225583967752755, "input_usage": {"receiver_prompt_tokens": 601}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600556544, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 2695874880, "process_rss_bytes": 6600556544}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
12 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "17e7e463c5868b352e0a43d6", "attempt_id": "b1a1be404836", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.546022Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1971}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 598, "message_tokens": 516, "message_utf8_bytes": 1775, "cache_tensor_bytes": null, "serialized_payload_bytes": 1775, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.7649579094722867, "decode_payload_ms": 0.0, "receiver_prefill_ms": 95.2667499659583, "ttft_ms": 23.73883395921439, "decode_ms": 22.320416988804936}, "total_wall_ms": 142.76070904452354, "ttft_ms": 23.73883395921439, "input_usage": {"receiver_prompt_tokens": 598}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600654848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6568931416, "mlx_cache_bytes": 2941831488, "process_rss_bytes": 6600654848}, "peak_prefill_memory": null, "peak_decode_memory": 6568931416, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "d2f5c011b1bc04cde05fd53e8eda8cce4f41dc708c77ba1e0b49bd3fb47d3958", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
13 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4e1e37e59f7beee987b1ca29", "attempt_id": "ff5f09c23dec", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.712474Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 583, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 92.41145791020244, "ttft_ms": 21.56287501566112, "decode_ms": 15.641708974726498}, "total_wall_ms": 130.70283306296915, "ttft_ms": 21.56287501566112, "input_usage": {"receiver_prompt_tokens": 583}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600769536, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6559592536, "mlx_cache_bytes": 3296414016, "process_rss_bytes": 6600769536}, "peak_prefill_memory": null, "peak_decode_memory": 6559592536, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
14 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "12de52603bea6e9934f5b230", "attempt_id": "71f4c1fad7f1", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:39.862544Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 589, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 93.39258295949548, "ttft_ms": 23.743500001728535, "decode_ms": 20.554958027787507}, "total_wall_ms": 139.25754197407514, "ttft_ms": 23.743500001728535, "input_usage": {"receiver_prompt_tokens": 589}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6600835072, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6563131480, "mlx_cache_bytes": 3639789888, "process_rss_bytes": 6600835072}, "peak_prefill_memory": null, "peak_decode_memory": 6563131480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
15 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a5db44064cdbeec9c2302287", "attempt_id": "6e66fa4e2bab", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.022475Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 589, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 92.00699999928474, "ttft_ms": 21.959374891594052, "decode_ms": 176.00433400366455}, "total_wall_ms": 291.4037499576807, "ttft_ms": 21.959374891594052, "input_usage": {"receiver_prompt_tokens": 589}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600835072, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6563131480, "mlx_cache_bytes": 3639789888, "process_rss_bytes": 6600835072}, "peak_prefill_memory": null, "peak_decode_memory": 6563131480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,068,030", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "d1643b9a5cdbde943cffa17e0bd2ea3a63bd11939d2b80e5c53fe3f1d39a8e26"}
16 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "042990c4fab2319bcaa2df57", "attempt_id": "6f2aeaacc7b7", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.333810Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 14}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 586, "message_tokens": 508, "message_utf8_bytes": 1231, "cache_tensor_bytes": null, "serialized_payload_bytes": 1231, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.582667046226561, "decode_payload_ms": 0.0, "receiver_prefill_ms": 88.77624990418553, "ttft_ms": 21.344291046261787, "decode_ms": 102.45566698722541}, "total_wall_ms": 213.67987501434982, "ttft_ms": 21.344291046261787, "input_usage": {"receiver_prompt_tokens": 586}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6600851456, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6561951832, "mlx_cache_bytes": 3880831296, "process_rss_bytes": 6600851456}, "peak_prefill_memory": null, "peak_decode_memory": 6561951832, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "386755a6041624133cae3e906274e1dbaa1ac732378d43f385e9581f5c9aba44", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
17 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "8c5856e58a4069eedbaf6563", "attempt_id": "c4a0944c90e1", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.568489Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 512, "n_sentences": 24}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 587, "message_tokens": 512, "message_utf8_bytes": 1358, "cache_tensor_bytes": null, "serialized_payload_bytes": 1358, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.1340419296175241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 89.60275002755225, "ttft_ms": 20.019917050376534, "decode_ms": 194.40183392725885}, "total_wall_ms": 306.6186250653118, "ttft_ms": 20.019917050376534, "input_usage": {"receiver_prompt_tokens": 587}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6600900608, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6562246744, "mlx_cache_bytes": 4122167616, "process_rss_bytes": 6600900608}, "peak_prefill_memory": null, "peak_decode_memory": 6562246744, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "9c6f1d7683c25b99905915cd50af46f8624d0e046d60064afa1f8536d2053414", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
18 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "d2d4bfa6f46817c4a6046d3b", "attempt_id": "3a3af24c7c55", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:40.895456Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 25}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 589, "message_tokens": 508, "message_utf8_bytes": 1477, "cache_tensor_bytes": null, "serialized_payload_bytes": 1477, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.0682919528335333, "decode_payload_ms": 0.0, "receiver_prefill_ms": 80.28387499507517, "ttft_ms": 20.092083024792373, "decode_ms": 186.3948330283165}, "total_wall_ms": 289.03933404944837, "ttft_ms": 20.092083024792373, "input_usage": {"receiver_prompt_tokens": 589}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6600916992, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6563131480, "mlx_cache_bytes": 4122167616, "process_rss_bytes": 6600916992}, "peak_prefill_memory": null, "peak_decode_memory": 6563131480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0c282c4bd4288101fb9f0b7c3f7960301f9b9cae0e1b006f3d55fa33cb6c3bdc", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
19 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "46b4c199d55ef7f74812ec8b", "attempt_id": "7ae1fd080653", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:41.203354Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 499, "n_sentences": 24}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 580, "message_tokens": 499, "message_utf8_bytes": 1320, "cache_tensor_bytes": null, "serialized_payload_bytes": 1320, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.1374169262126088, "decode_payload_ms": 0.0, "receiver_prefill_ms": 96.964625059627, "ttft_ms": 15.490291989408433, "decode_ms": 136.05291699059308}, "total_wall_ms": 250.9514580015093, "ttft_ms": 15.490291989408433, "input_usage": {"receiver_prompt_tokens": 580}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6600916992, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6577401944, "mlx_cache_bytes": 4382132544, "process_rss_bytes": 6600916992}, "peak_prefill_memory": null, "peak_decode_memory": 6577401944, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "77c2c83f97318ad674e337c84e8706903022a99e330a15111b84b24346998e7c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
20 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2919933128528059f21bcef7", "attempt_id": "bac3b7a88e92", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:41.474626Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 499, "n_sentences": 17}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 577, "message_tokens": 499, "message_utf8_bytes": 1439, "cache_tensor_bytes": null, "serialized_payload_bytes": 1439, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.8985829772427678, "decode_payload_ms": 0.0, "receiver_prefill_ms": 201.23016694560647, "ttft_ms": 18.682542024180293, "decode_ms": 100.2345000160858}, "total_wall_ms": 322.1747080096975, "ttft_ms": 18.682542024180293, "input_usage": {"receiver_prompt_tokens": 577}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6600982528, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6555218008, "mlx_cache_bytes": 4697098560, "process_rss_bytes": 6600982528}, "peak_prefill_memory": null, "peak_decode_memory": 6555218008, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "8c78c2c57027b9340eda90e83f732ddd57af36c5177bd2241b5f21f5452cfb30", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
21 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c9f341728f8327673d8c20fd", "attempt_id": "75d22a7bf90c", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:52.355810Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 591, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 96.65604203473777, "ttft_ms": 21.105334046296775, "decode_ms": 193.9961250172928}, "total_wall_ms": 313.1064580520615, "ttft_ms": 21.105334046296775, "input_usage": {"receiver_prompt_tokens": 591, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6601342976, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6584578136, "mlx_cache_bytes": 5159553344, "process_rss_bytes": 6601342976}, "peak_prefill_memory": null, "peak_decode_memory": 6584578136, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
22 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3d4dd464f7010b36c96bc271", "attempt_id": "77a451d8dea8", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:52.688816Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 99.47799996007234, "ttft_ms": 24.71804094966501, "decode_ms": 19.204750075004995}, "total_wall_ms": 144.63758398778737, "ttft_ms": 24.71804094966501, "input_usage": {"receiver_prompt_tokens": 597, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601375744, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 5404920128, "process_rss_bytes": 6601375744}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
23 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0a2034eccdf34fd25393da3d", "attempt_id": "125915e60a3b", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:52.855392Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 104.98479206580669, "ttft_ms": 21.320041036233306, "decode_ms": 13.582999934442341}, "total_wall_ms": 141.07616699766368, "ttft_ms": 21.320041036233306, "input_usage": {"receiver_prompt_tokens": 597, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601375744, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6567653464, "mlx_cache_bytes": 5404920128, "process_rss_bytes": 6601375744}, "peak_prefill_memory": null, "peak_decode_memory": 6567653464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
24 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e8e8d23e7cdbfbbae9a8842f", "attempt_id": "1a19d08c27d6", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:26:53.016421Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10538.946290966123, "decode_payload_ms": 0.0, "receiver_prefill_ms": 98.61950005870312, "ttft_ms": 26.502083986997604, "decode_ms": 22.947457968257368}, "total_wall_ms": 149.47245793882757, "ttft_ms": 26.502083986997604, "input_usage": {"receiver_prompt_tokens": 594, "sender_input_tokens": 2087, "sender_output_tokens": 512, "sender_prefill_ms": 310.1318330736831, "sender_ttft_ms": 21.124709048308432, "sender_decode_ms": 10205.545666976832}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601375744, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6566572120, "mlx_cache_bytes": 5561616704, "process_rss_bytes": 6601375744}, "peak_prefill_memory": null, "peak_decode_memory": 6566572120, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
25 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bfe20256c44e4c579db64576", "attempt_id": "1297d741734b", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:03.898439Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 434, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 86.80112496949732, "ttft_ms": 15.794583945535123, "decode_ms": 15.125207952223718}, "total_wall_ms": 119.22554101329297, "ttft_ms": 15.794583945535123, "input_usage": {"receiver_prompt_tokens": 434, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601621504, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6536062772, "mlx_cache_bytes": 5921099780, "process_rss_bytes": 6601621504}, "peak_prefill_memory": null, "peak_decode_memory": 6536062772, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
26 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "14528bf1cb04b7494a446b9d", "attempt_id": "2940c8ef0c3d", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:04.040109Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 440, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 80.46558406203985, "ttft_ms": 26.1877499287948, "decode_ms": 23.025874979794025}, "total_wall_ms": 130.96516602672637, "ttft_ms": 26.1877499287948, "input_usage": {"receiver_prompt_tokens": 440, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601703424, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6540175180, "mlx_cache_bytes": 6264870624, "process_rss_bytes": 6601703424}, "peak_prefill_memory": null, "peak_decode_memory": 6540175180, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
27 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ad104e7f7b140ccf40a30520", "attempt_id": "f9ea7cc2404d", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:04.197844Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 440, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 84.51795799192041, "ttft_ms": 22.2842920338735, "decode_ms": 20.577540970407426}, "total_wall_ms": 128.7586250109598, "ttft_ms": 22.2842920338735, "input_usage": {"receiver_prompt_tokens": 440, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601736192, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6540175180, "mlx_cache_bytes": 6264870624, "process_rss_bytes": 6601736192}, "peak_prefill_memory": null, "peak_decode_memory": 6540175180, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
28 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0d6fd2ea1c04214d97935766", "attempt_id": "0efbda3ef5a6", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:04.350742Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 437, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10709.832667023875, "decode_payload_ms": 0.0, "receiver_prefill_ms": 73.29770794603974, "ttft_ms": 22.63987495098263, "decode_ms": 22.071832907386124}, "total_wall_ms": 119.35541604179889, "ttft_ms": 22.63987495098263, "input_usage": {"receiver_prompt_tokens": 437, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 363.058082992211, "sender_ttft_ms": 21.328666945919394, "sender_decode_ms": 10323.293083929457}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601736192, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6537635660, "mlx_cache_bytes": 6504208096, "process_rss_bytes": 6601736192}, "peak_prefill_memory": null, "peak_decode_memory": 6537635660, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 8, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
29 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "cc55ee727e706809191c6b97", "attempt_id": "18419046d12f", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.157932Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1045, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 158.85141701437533, "ttft_ms": 23.864249931648374, "decode_ms": 21.887833951041102}, "total_wall_ms": 206.70400001108646, "ttft_ms": 23.864249931648374, "input_usage": {"receiver_prompt_tokens": 1045, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601932800, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6818152508, "mlx_cache_bytes": 7174788832, "process_rss_bytes": 6601932800}, "peak_prefill_memory": null, "peak_decode_memory": 6818152508, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "561d3553701d834273477e7e81d6f4a2903e1fe9e09b5b4efdd73a721bde815e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
30 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "501d065cba8addb831258e6e", "attempt_id": "de2d1ad7bd00", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.386259Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1051, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 189.56966698169708, "ttft_ms": 14.294749940745533, "decode_ms": 27.563792071305215}, "total_wall_ms": 233.33387507591397, "ttft_ms": 14.294749940745533, "input_usage": {"receiver_prompt_tokens": 1051, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6601998336, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6821855292, "mlx_cache_bytes": 7792318176, "process_rss_bytes": 6601998336}, "peak_prefill_memory": null, "peak_decode_memory": 6821855292, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ab0e9c7021e297c7521daf9f4fa54de8136f15dc9e1c13a37a4af71f57687c8e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
31 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f765949413aa3fedfc527088", "attempt_id": "ceca82621c8d", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.642557Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1051, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 189.3771659815684, "ttft_ms": 25.745707913301885, "decode_ms": 17.177957925014198}, "total_wall_ms": 234.096874948591, "ttft_ms": 25.745707913301885, "input_usage": {"receiver_prompt_tokens": 1051, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6602014720, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6821855292, "mlx_cache_bytes": 7792318176, "process_rss_bytes": 6602014720}, "peak_prefill_memory": null, "peak_decode_memory": 6821855292, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ab0e9c7021e297c7521daf9f4fa54de8136f15dc9e1c13a37a4af71f57687c8e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
32 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "9b842e322458a64162511f40", "attempt_id": "c420975d5b17", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:15.899962Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 7}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 1048, "message_tokens": 970, "message_utf8_bytes": 623, "cache_tensor_bytes": null, "serialized_payload_bytes": 623, "side_information_bytes": 0, "fetched_bytes": 1634, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10663.693792070262, "decode_payload_ms": 0.0, "receiver_prefill_ms": 191.9341660104692, "ttft_ms": 21.331749972887337, "decode_ms": 21.82987506967038}, "total_wall_ms": 236.95158294867724, "ttft_ms": 21.331749972887337, "input_usage": {"receiver_prompt_tokens": 1048, "sender_input_tokens": 2115, "sender_output_tokens": 512, "sender_prefill_ms": 357.24162496626377, "sender_ttft_ms": 24.779249913990498, "sender_decode_ms": 10279.268582933582}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6602031104, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6850740284, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602031104}, "peak_prefill_memory": null, "peak_decode_memory": 6850740284, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1634, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ab0e9c7021e297c7521daf9f4fa54de8136f15dc9e1c13a37a4af71f57687c8e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "dbb4d6f308be12c0c7abcf21", "attempt_id": "0f02e576dce2", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:16.163883Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2045, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 351.0584579780698, "ttft_ms": 25.52862500306219, "decode_ms": 203.8207920268178}, "total_wall_ms": 583.828458096832, "ttft_ms": 25.52862500306219, "input_usage": {"receiver_prompt_tokens": 2045}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6602473472, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7061651504, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602473472}, "peak_prefill_memory": null, "peak_decode_memory": 7061651504, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
34 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a05e1aeb0d35317b7f3245e7", "attempt_id": "5ae486a7ef40", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:16.769440Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 321.8972079921514, "ttft_ms": 22.83312496729195, "decode_ms": 189.30195807479322}, "total_wall_ms": 537.459957995452, "ttft_ms": 22.83312496729195, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6602522624, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602522624}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
35 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a2c9f22c371cea273080a863", "attempt_id": "9bc42d166be3", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:17.329219Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2051, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 320.60200010892004, "ttft_ms": 21.40745904762298, "decode_ms": 157.95058303046972}, "total_wall_ms": 503.5944579867646, "ttft_ms": 21.40745904762298, "input_usage": {"receiver_prompt_tokens": 2051}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6602522624, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054786612, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602522624}, "peak_prefill_memory": null, "peak_decode_memory": 7054786612, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
36 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0f75b3b41655640fb015977f", "attempt_id": "ce4f9d61ae8a", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:17.855898Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 2048, "message_tokens": 1971, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5.297624971717596, "decode_payload_ms": 0.417957897298038, "receiver_prefill_ms": 304.32729201857, "ttft_ms": 20.300709060393274, "decode_ms": 107.78937500435859}, "total_wall_ms": 435.70933397859335, "ttft_ms": 20.300709060393274, "input_usage": {"receiver_prompt_tokens": 2048}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6602539008, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7063175232, "mlx_cache_bytes": 8266307296, "process_rss_bytes": 6602539008}, "peak_prefill_memory": null, "peak_decode_memory": 7063175232, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
37 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c5c8f77106a5a3772c8729a5", "attempt_id": "4c8b0eb2ff80", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:18.659274Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 8.98820802103728, "receiver_prefill_ms": 0.0, "query_prefill_ms": 249.4774169754237, "ttft_ms": 249.4774169754237, "decode_ms": 216.42729197628796}, "total_wall_ms": 477.1647499874234, "ttft_ms": 249.4774169754237, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6751551488, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6445818172, "mlx_cache_bytes": 9171015533, "process_rss_bytes": 6751551488}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6445818172, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
38 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "36b870572f370219b55f990f", "attempt_id": "90b0dceb3de2", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.158034Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 0.8970829658210278, "receiver_prefill_ms": 0.0, "query_prefill_ms": 42.57166699972004, "ttft_ms": 42.57166699972004, "decode_ms": 185.1583339739591}, "total_wall_ms": 230.58637499343604, "ttft_ms": 42.57166699972004, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6751666176, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6450323772, "mlx_cache_bytes": 9211369489, "process_rss_bytes": 6751666176}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450323772, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
39 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "5c39b27a1d42aae9403aeea4", "attempt_id": "39eeebfa218d", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.410787Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 0.850292039103806, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.38766698632389, "ttft_ms": 32.38766698632389, "decode_ms": 154.78104201611131}, "total_wall_ms": 190.03108399920166, "ttft_ms": 32.38766698632389, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6751666176, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6450323684, "mlx_cache_bytes": 9211369489, "process_rss_bytes": 6751666176}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450323684, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
40 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f2fea6b40fbfa3efc8b41254", "attempt_id": "6551be76c24d", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.622622Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321889288}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112716, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 346.22316691093147, "doc_prefill_ms": 308.5541669279337, "compress_ms": 9.487665956839919, "serialize_ms": 24.9150829622522, "load_ms": 0.896209036000073, "receiver_prefill_ms": 0.0, "query_prefill_ms": 208.85649998672307, "ttft_ms": 208.85649998672307, "decode_ms": 107.41054196842015}, "total_wall_ms": 318.9865410095081, "ttft_ms": 208.85649998672307, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6751764480, "backend_memory": {"mlx_active_bytes": 6398566408, "mlx_peak_bytes": 6446309604, "mlx_cache_bytes": 9222920209, "process_rss_bytes": 6751764480}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6446309604, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0fd1e41ac9de2a394237f6d73e79f07db7814fc3daad26bdafb5187f0b9128ae", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
41 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "5242938fc112225287b86b4e", "attempt_id": "0439f90f11d2", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:19.986065Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 11.17420895025134, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.265333062037826, "ttft_ms": 31.265333062037826, "decode_ms": 209.63187504094094}, "total_wall_ms": 253.80166701506823, "ttft_ms": 31.265333062037826, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6826115072, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6445654220, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6826115072}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6445654220, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
42 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "7233daed51e30233779143b4", "attempt_id": "bc0068e8d567", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.262668Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 9.897915995679796, "receiver_prefill_ms": 0.0, "query_prefill_ms": 34.51216605026275, "ttft_ms": 34.51216605026275, "decode_ms": 170.15541705768555}, "total_wall_ms": 216.34687494952232, "ttft_ms": 34.51216605026275, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6832308224, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6450159844, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6832308224}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450159844, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
43 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "160b26eb5f87ef0ce84beb4d", "attempt_id": "c503e8d98716", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.502861Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 9.91812499705702, "receiver_prefill_ms": 0.0, "query_prefill_ms": 33.97083294112235, "ttft_ms": 33.97083294112235, "decode_ms": 151.9315839977935}, "total_wall_ms": 197.57162500172853, "ttft_ms": 33.97083294112235, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6838501376, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6450159844, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6838501376}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6450159844, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
44 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "00a9e3a43807646a1eff16b6", "attempt_id": "b124df1d7a76", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.724657Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6321692680}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "5934e1e7b2bd92f6", "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 74096640, "serialized_payload_bytes": 74112744, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21.857209037989378, "doc_prefill_ms": 308.5541669279337, "compress_ms": 1.7394160386174917, "serialize_ms": 19.88820801489055, "load_ms": 7.181875058449805, "receiver_prefill_ms": 0.0, "query_prefill_ms": 38.916666992008686, "ttft_ms": 38.916666992008686, "decode_ms": 104.37366692349315}, "total_wall_ms": 152.26829203311354, "ttft_ms": 38.916666992008686, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6850887680, "backend_memory": {"mlx_active_bytes": 6398369800, "mlx_peak_bytes": 6446145764, "mlx_cache_bytes": 9223116817, "process_rss_bytes": 6850887680}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6446145764, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "eb4d9d05bd93beebadeb0b446aca6f8cd3f69a85009c57c2686509661e53bad1", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
45 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "294ffec3fe617d6816ea913d", "attempt_id": "1828d4dd5c48", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:20.938893Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 12.6062500057742, "receiver_prefill_ms": 0.0, "query_prefill_ms": 273.0062920600176, "ttft_ms": 273.0062920600176, "decode_ms": 218.48266595043242}, "total_wall_ms": 505.92758296988904, "ttft_ms": 273.0062920600176, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6891094016, "backend_memory": {"mlx_active_bytes": 6331681384, "mlx_peak_bytes": 6388051512, "mlx_cache_bytes": 9450675425, "process_rss_bytes": 6891094016}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6388051512, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
46 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bcbcd6fa055fead9c2aa2d75", "attempt_id": "1b94f96b3250", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:21.469067Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 1.5839169500395656, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.846250005997717, "ttft_ms": 25.846250005997717, "decode_ms": 186.94529205095023}, "total_wall_ms": 216.11754200421274, "ttft_ms": 25.846250005997717, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6891143168, "backend_memory": {"mlx_active_bytes": 6331681384, "mlx_peak_bytes": 6397167472, "mlx_cache_bytes": 9476273137, "process_rss_bytes": 6891143168}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6397167472, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
47 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ad87548daad905473e842531", "attempt_id": "507a3aee8516", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:21.707191Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 1.4031250029802322, "receiver_prefill_ms": 0.0, "query_prefill_ms": 26.463374961167574, "ttft_ms": 26.463374961167574, "decode_ms": 176.86091596260667}, "total_wall_ms": 206.55783300753683, "ttft_ms": 26.463374961167574, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6891143168, "backend_memory": {"mlx_active_bytes": 6331665000, "mlx_peak_bytes": 6397168208, "mlx_cache_bytes": 9476289521, "process_rss_bytes": 6891143168}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6397168208, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
48 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "131a6411d0b38a14e350647f", "attempt_id": "72228fa64660", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:21.935900Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6287739496}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 39363840, "serialized_payload_bytes": 39398030, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 39.9517499608919, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.196083027869463, "serialize_ms": 16.328124911524355, "load_ms": 1.620083930902183, "receiver_prefill_ms": 0.0, "query_prefill_ms": 180.2529999986291, "ttft_ms": 180.2529999986291, "decode_ms": 113.61583299003541}, "total_wall_ms": 297.15695802588016, "ttft_ms": 180.2529999986291, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6891192320, "backend_memory": {"mlx_active_bytes": 6331681384, "mlx_peak_bytes": 6389402192, "mlx_cache_bytes": 9498686449, "process_rss_bytes": 6891192320}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6389402192, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5f8624f35d651cd19bdb054c3205083465b76bee66103dc2a4d23dbd2e6dc167", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
49 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "cfa6d276303301b1b0a340f3", "attempt_id": "3ce90c85cbe1", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:22.289140Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 6.075291079469025, "receiver_prefill_ms": 0.0, "query_prefill_ms": 259.889583918266, "ttft_ms": 259.889583918266, "decode_ms": 1147.461749962531}, "total_wall_ms": 1415.1050000218675, "ttft_ms": 259.889583918266, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6912704512, "backend_memory": {"mlx_active_bytes": 6294484232, "mlx_peak_bytes": 6346778920, "mlx_cache_bytes": 9564606801, "process_rss_bytes": 6912704512}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6346778920, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "\n\nunknownunknownunknown memo\n\n5\nunknown memo\n5\n\n\n memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo memo", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "78d55bf7eaf8f5d5718fae1592c5390a9d4187e68a847a5372b8f5fb9e43768c"}
50 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "34a0a1b53fd1dae009867c20", "attempt_id": "0d83c3b1bfcd", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:23.727440Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 2.2449579555541277, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.74408390279859, "ttft_ms": 32.74408390279859, "decode_ms": 1060.114708961919}, "total_wall_ms": 1096.9503748929128, "ttft_ms": 32.74408390279859, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6912737280, "backend_memory": {"mlx_active_bytes": 6294549768, "mlx_peak_bytes": 6356190592, "mlx_cache_bytes": 9564541265, "process_rss_bytes": 6912737280}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6356190592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": " \nuser\n\n memo20.\nat000000000 of000000000000000000000000 of000", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "f956396894d647df89bc96127bf3812773aa370217fc6449ccdda4300087a89f"}
51 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a8b57b89f00aa8409a780a6d", "attempt_id": "65169d0f70f2", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:24.846333Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 1.5409999759867787, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.786499958485365, "ttft_ms": 25.786499958485365, "decode_ms": 55.565542075783014}, "total_wall_ms": 84.65270895976573, "ttft_ms": 25.786499958485365, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6912737280, "backend_memory": {"mlx_active_bytes": 6294566152, "mlx_peak_bytes": 6356190592, "mlx_cache_bytes": 9564524881, "process_rss_bytes": 6912737280}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6356190592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "\n\n", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "invalid_output", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "75a11da44c802486bc6f65640aa48a730f0f684c5c07a42ba3cd1735eb3fb070"}
52 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "01f04fced02030ea8b7fb96c", "attempt_id": "91a614851ec3", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:24.952208Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268712200}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 20839680, "serialized_payload_bytes": 20873736, "side_information_bytes": 34056, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 34.626082982867956, "doc_prefill_ms": 308.5541669279337, "compress_ms": 23.7636249512434, "serialize_ms": 10.439375066198409, "load_ms": 1.630165963433683, "receiver_prefill_ms": 0.0, "query_prefill_ms": 180.5562499212101, "ttft_ms": 180.5562499212101, "decode_ms": 1070.0649589998648}, "total_wall_ms": 1254.2412499897182, "ttft_ms": 180.5562499212101, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6912786432, "backend_memory": {"mlx_active_bytes": 6294500616, "mlx_peak_bytes": 6348424592, "mlx_cache_bytes": 9564590417, "process_rss_bytes": 6912786432}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6348424592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "user\n\nsystem 0 0 000000000000000000000000000000000000000", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "76d58caa4868c5c31b8703292423da4d64fc4061ef7af423b7b0c2b1f80e4f14", "output_hash": "d3da647a46cbd3eb57b7ba22bfc90fff80e5d0dfb8efe06d0ea15bf1b7249bde"}
53 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3866a433f650f0db81383698", "attempt_id": "94d40d88413a", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.301231Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 1.023292075842619, "receiver_prefill_ms": 0.0, "query_prefill_ms": 35.569333005696535, "ttft_ms": 35.569333005696535, "decode_ms": 199.1004589945078}, "total_wall_ms": 237.6932919723913, "ttft_ms": 35.569333005696535, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6914555904, "backend_memory": {"mlx_active_bytes": 6287777800, "mlx_peak_bytes": 6324363468, "mlx_cache_bytes": 9575183873, "process_rss_bytes": 6914555904}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6324363468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "MP-89X-6320", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "e07259f8346a7570e5b980e01a868e6af8ede517e6d7f6a2958e2f3b8da08023"}
54 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c5a46a2a80d968eb5b9cfd03", "attempt_id": "d3ca7d571fc6", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.559533Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 0.7745410548523068, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.40566700231284, "ttft_ms": 27.40566700231284, "decode_ms": 165.24295799899846}, "total_wall_ms": 195.35333395469934, "ttft_ms": 27.40566700231284, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6914588672, "backend_memory": {"mlx_active_bytes": 6287777800, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9575183873, "process_rss_bytes": 6914588672}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
55 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "9e9adefa1df3c8162ef90094", "attempt_id": "d4d46f7011e0", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.777798Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 0.9067920036613941, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.181916986592114, "ttft_ms": 27.181916986592114, "decode_ms": 32.393874949775636}, "total_wall_ms": 62.11195804644376, "ttft_ms": 27.181916986592114, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6914588672, "backend_memory": {"mlx_active_bytes": 6287745032, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9575216641, "process_rss_bytes": 6914588672}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not provided", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "d83b9ff3c07a8390e2ea0a2af45fba37595da25f31b85ba625363ee98c0c796f"}
56 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a4e24a1d6a957db1e957f87c", "attempt_id": "642d96719fa3", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:26.863105Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 64.08412498421967}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521645, "side_information_bytes": 15917, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.80583295505494, "doc_prefill_ms": 308.5541669279337, "compress_ms": 64.13950002752244, "serialize_ms": 8.448374923318624, "load_ms": 1.1591659858822823, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.929875068366528, "ttft_ms": 27.929875068366528, "decode_ms": 76.25045790337026}, "total_wall_ms": 106.98124999180436, "ttft_ms": 27.929875068366528, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6914588672, "backend_memory": {"mlx_active_bytes": 6287155208, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9575806465, "process_rss_bytes": 6914588672}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Milos Ferrante", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7f6ef81007573cfaec410eb9847c70ee516bcb552b3345400893355ca501b96b", "output_hash": "14ae3adaefbe5d307a9585834a37951eb0794dc1bfacd0739a580886526143d2"}
57 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c5299d0a9079f76c811be8e1", "attempt_id": "a91029007c95", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:27.021383Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.7211669581010938, "receiver_prefill_ms": 0.0, "query_prefill_ms": 22.886250051669776, "ttft_ms": 22.886250051669776, "decode_ms": 941.9688330963254}, "total_wall_ms": 967.6620420068502, "ttft_ms": 22.886250051669776, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934003712, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6324363468, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934003712}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6324363468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant, Assistant\n\nAssistant, Assistant,", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "a7f3f386f2ebe03f3097faba3f505d405d26b2f9abd5b6faacedc39b50901a2d"}
58 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "92833db31d27eea8ff70489c", "attempt_id": "7e0b4656c29b", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:28.012306Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.9810830233618617, "receiver_prefill_ms": 0.0, "query_prefill_ms": 28.88324996456504, "ttft_ms": 28.88324996456504, "decode_ms": 929.5173750724643}, "total_wall_ms": 961.1695419298485, "ttft_ms": 28.88324996456504, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934020096, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934020096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a memo. I am a", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "426a0f227def734f39a983ae88dcd05e8dcbe992627914324fb637eb8cb63e6b"}
59 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2cbc1b12a43668b7e57b864c", "attempt_id": "0d1f38b0b06c", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:28.997109Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.652667018584907, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.082583030685782, "ttft_ms": 29.082583030685782, "decode_ms": 957.0584170287475}, "total_wall_ms": 988.5560829425231, "ttft_ms": 29.082583030685782, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934020096, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934020096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "The Project codes are the name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name name", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "79576cc33fa68a3208636b2d8810c8c0d1f2117032776dc346a524e04fb85a25"}
60 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "169b50740e3460318aa49fa9", "attempt_id": "e99d3d382fb0", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:30.008009Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 2010, "keep": 502, "compress_ms": 25.0343750230968}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521639, "side_information_bytes": 15911, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 31.907333061099052, "doc_prefill_ms": 308.5541669279337, "compress_ms": 25.070416973903775, "serialize_ms": 6.617291015572846, "load_ms": 0.8759999182075262, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.01162502914667, "ttft_ms": 30.01162502914667, "decode_ms": 949.0018750075251}, "total_wall_ms": 981.6403749864548, "ttft_ms": 30.01162502914667, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6934020096, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9596089857, "process_rss_bytes": 6934020096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "I will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\nI will revise the project.\n", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "367ec33700beb9aed70dd5a013f541cdebb9460e0b911a49ddd42cd63c849f4b", "output_hash": "ec85516726e09bae04746f5b6be22f47099830d26c5193e069713a8e4b00dca5"}
61 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "93f6f7676d8794e607f42e04", "attempt_id": "da4d2710c0b7", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:31.010413Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 35, "compress_ms": 563.8788749929518}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 572.4881670903414, "doc_prefill_ms": 308.5541669279337, "compress_ms": 565.5872919596732, "serialize_ms": 6.614041980355978, "load_ms": 0.8810000726953149, "receiver_prefill_ms": 0.0, "query_prefill_ms": 20.365500007756054, "ttft_ms": 20.365500007756054, "decode_ms": 119.11770806182176}, "total_wall_ms": 714.6071669412777, "ttft_ms": 20.365500007756054, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": 6936444928, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6322151628, "mlx_cache_bytes": 9625236993, "process_rss_bytes": 6936444928}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6322151628, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-1482", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "f5e9a6d78321ae31e0a3bdeeaf8c39b796c1fc12a0a8896955e8c411f428367c"}
62 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2f2d1a37db90019896731035", "attempt_id": "be106ee31fdd", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:31.746833Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 41, "compress_ms": 55.423791985958815}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521686, "side_information_bytes": 15958, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.9762079808861, "doc_prefill_ms": 308.5541669279337, "compress_ms": 57.0506660733372, "serialize_ms": 6.670959061011672, "load_ms": 0.9055420523509383, "receiver_prefill_ms": 0.0, "query_prefill_ms": 28.734207968227565, "ttft_ms": 28.734207968227565, "decode_ms": 179.1473749326542}, "total_wall_ms": 274.67562502715737, "ttft_ms": 28.734207968227565, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6936526848, "backend_memory": {"mlx_active_bytes": 6287777800, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9648109057, "process_rss_bytes": 6936526848}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2133e2d8fb6fbe8bb2bc071e98a75ef68a33983b8f86299196b1d6fddcd4bf21", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
63 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3c7f1c745d25a93cc74dfa5b", "attempt_id": "ea7dbd066ab8", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.046152Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 41, "compress_ms": 60.98887498956174}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 69.41050000023097, "doc_prefill_ms": 308.5541669279337, "compress_ms": 62.77983298059553, "serialize_ms": 6.343499990180135, "load_ms": 0.7366250501945615, "receiver_prefill_ms": 0.0, "query_prefill_ms": 28.80037494469434, "ttft_ms": 28.80037494469434, "decode_ms": 144.29624995682389}, "total_wall_ms": 245.3072089701891, "ttft_ms": 28.80037494469434, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6936559616, "backend_memory": {"mlx_active_bytes": 6287745032, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9650468353, "process_rss_bytes": 6936559616}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
64 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c8f2e4a96d7379d7d34702d1", "attempt_id": "e9277718bcd6", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.313036Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 38, "compress_ms": 175.28775008395314}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521686, "side_information_bytes": 15958, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 182.38962499890476, "doc_prefill_ms": 308.5541669279337, "compress_ms": 176.8307499587536, "serialize_ms": 5.288459011353552, "load_ms": 0.5769999697804451, "receiver_prefill_ms": 0.0, "query_prefill_ms": 26.420791051350534, "ttft_ms": 26.420791051350534, "decode_ms": 105.64537497702986}, "total_wall_ms": 316.87600002624094, "ttft_ms": 26.420791051350534, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6936657920, "backend_memory": {"mlx_active_bytes": 6287745032, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9655367169, "process_rss_bytes": 6936657920}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2133e2d8fb6fbe8bb2bc071e98a75ef68a33983b8f86299196b1d6fddcd4bf21", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
65 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a0f56d97be7daffa8cef249e", "attempt_id": "3e0f81469768", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.713987Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 1.1760419001802802, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.72891598660499, "ttft_ms": 31.72891598660499, "decode_ms": 19.485582946799695}, "total_wall_ms": 54.12295798305422, "ttft_ms": 31.72891598660499, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936887296, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6322151628, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936887296}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6322151628, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
66 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "490a5844ac7e4eb254afa577", "attempt_id": "24d3c5e9fd52", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.789602Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 0.7209168979898095, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.789833045564592, "ttft_ms": 25.789833045564592, "decode_ms": 24.353457964025438}, "total_wall_ms": 52.51691699959338, "ttft_ms": 25.789833045564592, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936920064, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936920064}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
67 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f43fe1d7913c24ca7775f970", "attempt_id": "a9cecb8c4957", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.863379Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 0.816666055470705, "receiver_prefill_ms": 0.0, "query_prefill_ms": 19.551209057681262, "ttft_ms": 19.551209057681262, "decode_ms": 23.046249989420176}, "total_wall_ms": 45.09937507100403, "ttft_ms": 19.551209057681262, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936920064, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6327771364, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936920064}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6327771364, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
68 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "23953c62f9522d607c97ebc1", "attempt_id": "df96117d0a60", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.930619Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 2010, "keep": 502, "window_tokens": 19, "compress_ms": 58.06616693735123}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 18505728, "serialized_payload_bytes": 18521685, "side_information_bytes": 15957, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 63.082249951548874, "doc_prefill_ms": 308.5541669279337, "compress_ms": 58.22541704401374, "serialize_ms": 4.690499976277351, "load_ms": 1.2563749914988875, "receiver_prefill_ms": 0.0, "query_prefill_ms": 25.40870802477002, "ttft_ms": 25.40870802477002, "decode_ms": 13.515500002540648}, "total_wall_ms": 41.98029194958508, "ttft_ms": 25.40870802477002, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6936920064, "backend_memory": {"mlx_active_bytes": 6286860296, "mlx_peak_bytes": 6325493988, "mlx_cache_bytes": 9674978817, "process_rss_bytes": 6936920064}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6325493988, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c45b4f30b880b0f5227d71d6f0c6992840fbb19374bbf7cfe2bdd05e112c7db5", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
69 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e4c6ac1cbe2528e118fc5703", "attempt_id": "a857f215ff82", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:32.994458Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 77.96645897906274, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (35,966) and (1,16,35,942) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
70 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e7b9ff431123b343d517b9dc", "attempt_id": "900e4aabd23d", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.096831Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 66.20066694449633, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (41,972) and (1,16,41,948) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
71 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4447de7a1f609caa2fe38d9d", "attempt_id": "77c4b23465cf", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.184948Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 75.23312489502132, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (41,972) and (1,16,41,948) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
72 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "135f1a8ecad430fbb7dadf92", "attempt_id": "6f686bd52983", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.282090Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 71.89062493853271, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (38,969) and (1,16,38,945) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
73 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "56ee991b90464dd33a7c9ee5", "attempt_id": "f3c10dcea6ab", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.403424Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 35, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.8424169393256307, "receiver_prefill_ms": 0.0, "query_prefill_ms": 19.6460000006482, "ttft_ms": 19.6460000006482, "decode_ms": 52.42887500207871}, "total_wall_ms": 75.69141697604209, "ttft_ms": 19.6460000006482, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 35}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6983172096, "backend_memory": {"mlx_active_bytes": 6290394144, "mlx_peak_bytes": 6335647168, "mlx_cache_bytes": 9688339849, "process_rss_bytes": 6983172096}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6335647168, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not provided", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "d83b9ff3c07a8390e2ea0a2af45fba37595da25f31b85ba625363ee98c0c796f"}
74 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2f67a67e0c52d41f79c0f979", "attempt_id": "8b2fad1d3eca", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.501685Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.884708064608276, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.582791983149946, "ttft_ms": 27.582791983149946, "decode_ms": 213.58125004917383}, "total_wall_ms": 244.85879205167294, "ttft_ms": 27.582791983149946, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6983204864, "backend_memory": {"mlx_active_bytes": 6290656288, "mlx_peak_bytes": 6352739496, "mlx_cache_bytes": 9688077705, "process_rss_bytes": 6983204864}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6352739496, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "b6582a041b3d2b38ac6924099568ca2e6f1a610c6fb5f4de2e0516b90433bc2a"}
75 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "a457f580ccecc54298315139", "attempt_id": "5cf33b8c867a", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.767722Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 41, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.6323330346494913, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.149292036890984, "ttft_ms": 29.149292036890984, "decode_ms": 161.35833400767297}, "total_wall_ms": 194.04400000348687, "ttft_ms": 29.149292036890984, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 41}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6983204864, "backend_memory": {"mlx_active_bytes": 6290394144, "mlx_peak_bytes": 6352739496, "mlx_cache_bytes": 9688339849, "process_rss_bytes": 6983204864}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6352739496, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
76 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0a4e1364ef45cc38c5da4506", "attempt_id": "3b25f21b11c6", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:33.985086Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 2010, "keep": 1005, "compress_ms": 8.962583029642701, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1971, "source_tokens_receiver": 1971, "query_tokens": 38, "message_tokens": 2010, "message_utf8_bytes": 7280, "cache_tensor_bytes": 19681920, "serialized_payload_bytes": 19716076, "side_information_bytes": 34156, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 27.869292069226503, "doc_prefill_ms": 308.5541669279337, "compress_ms": 18.7781669665128, "serialize_ms": 8.690916933119297, "load_ms": 1.4646250056102872, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.344999977387488, "ttft_ms": 30.344999977387488, "decode_ms": 122.02841695398092}, "total_wall_ms": 155.62674996908754, "ttft_ms": 30.344999977387488, "input_usage": {"receiver_prefix_tokens": 2010, "dense_cache_bytes": 74096640, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6983204864, "backend_memory": {"mlx_active_bytes": 6290361376, "mlx_peak_bytes": 6340107432, "mlx_cache_bytes": 9699874185, "process_rss_bytes": 6983204864}, "peak_prefill_memory": 7048478768, "peak_decode_memory": 6340107432, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "7d69bc9e9fb081ba659e5e33c86d2bce4d329a5a3dbc4c317ae3342eb962c6d0", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
77 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b8d8e634e61d167ebc87181e", "attempt_id": "481c6ed42eac", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:34.241172Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 73, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011124997399747372, "decode_payload_ms": 0.0, "receiver_prefill_ms": 44.282374903559685, "ttft_ms": 19.709625048562884, "decode_ms": 20.926458993926644}, "total_wall_ms": 85.48883290495723, "ttft_ms": 19.709625048562884, "input_usage": {"receiver_prompt_tokens": 73}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6701006848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6245949864, "mlx_cache_bytes": 172412400, "process_rss_bytes": 6701006848}, "peak_prefill_memory": null, "peak_decode_memory": 6245949864, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:NameError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
78 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "826224a74ebf0be97179de8c", "attempt_id": "6aea9004d9cf", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:34.374957Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 2020, "message_tokens": 1937, "message_utf8_bytes": 8125, "cache_tensor_bytes": null, "serialized_payload_bytes": 8125, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.02150004729628563, "decode_payload_ms": 0.0, "receiver_prefill_ms": 295.2367920661345, "ttft_ms": 14.32908303104341, "decode_ms": 1010.4336669901386}, "total_wall_ms": 1323.3462499920279, "ttft_ms": 14.32908303104341, "input_usage": {"receiver_prompt_tokens": 2020}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701006848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7052050480, "mlx_cache_bytes": 1068830192, "process_rss_bytes": 6701006848}, "peak_prefill_memory": null, "peak_decode_memory": 7052050480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
79 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b723bddfa5449bee38ed407a", "attempt_id": "4a8fdefb533f", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:35.745633Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1937}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 604, "message_tokens": 516, "message_utf8_bytes": 2156, "cache_tensor_bytes": null, "serialized_payload_bytes": 2156, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.090957947075367, "decode_payload_ms": 0.0, "receiver_prefill_ms": 93.57858297880739, "ttft_ms": 23.513957974500954, "decode_ms": 911.6655830293894}, "total_wall_ms": 1030.1828749943525, "ttft_ms": 23.513957974500954, "input_usage": {"receiver_prompt_tokens": 604}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701006848, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6569799768, "mlx_cache_bytes": 1469386224, "process_rss_bytes": 6701006848}, "peak_prefill_memory": null, "peak_decode_memory": 6569799768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['region'] != 'Sud' and row['units'] >= 2 and row['shipped_days'] <= ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "5668e3e1f062ec6fbeef2cd13bd6309d297d591e881d4bbc87cf51b2a5cfaf9f", "output_hash": "49013de476d48dbb78217dba376506dfae969e7e8e9e11415afad7ea2f53b479"}
80 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "162117ede3591cddd228532b", "attempt_id": "e08d4687822b", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:36.823140Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 500, "n_sentences": 3}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 584, "message_tokens": 499, "message_utf8_bytes": 1474, "cache_tensor_bytes": null, "serialized_payload_bytes": 1474, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.0204579923301935, "decode_payload_ms": 0.0, "receiver_prefill_ms": 92.08316600415856, "ttft_ms": 20.726916030980647, "decode_ms": 923.1244999682531}, "total_wall_ms": 1037.3579170554876, "ttft_ms": 20.726916030980647, "input_usage": {"receiver_prompt_tokens": 584}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701056000, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6559985752, "mlx_cache_bytes": 1824361968, "process_rss_bytes": 6701056000}, "peak_prefill_memory": null, "peak_decode_memory": 6559985752, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['region'] == 'Sud' or row['units'] < 2 or row['returned']:\n continue\n ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "0d0b68d8ccb32536ba23b7d25e38a4021a853137ff08f20815b4c3d2b9b45d64", "output_hash": "99475af42ec047e0ce1a7d674ae59578eb16d871e0e2d91661b164982ead65f8"}
81 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "146dc78b2613ad3e8f9ca821", "attempt_id": "6654ee0f823d", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:37.904166Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 506, "n_sentences": 22}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 590, "message_tokens": 506, "message_utf8_bytes": 2318, "cache_tensor_bytes": null, "serialized_payload_bytes": 2318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.4914999483153224, "decode_payload_ms": 0.0, "receiver_prefill_ms": 98.94125000573695, "ttft_ms": 16.27620798535645, "decode_ms": 934.4471250660717}, "total_wall_ms": 1052.851833985187, "ttft_ms": 16.27620798535645, "input_usage": {"receiver_prompt_tokens": 590}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701056000, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6564016216, "mlx_cache_bytes": 2183368176, "process_rss_bytes": 6701056000}, "peak_prefill_memory": null, "peak_decode_memory": 6564016216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for r in rows:\n if r['region'] == 'Sud' or r['units'] < 2 or r['shipped_days'] > ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "32a2539d2b6ee4c06cee34a3497e76891f4e908fa56a9f3359ae6958be2a1961", "output_hash": "740d0b6993023215fc488fca5404cbe262f0405d7e3192da63ee24a24b6a699b"}
82 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "6f8802cbd5053f88195deae0", "attempt_id": "ee7348e705c7", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:49.622450Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 600, "message_tokens": 512, "message_utf8_bytes": 1493, "cache_tensor_bytes": null, "serialized_payload_bytes": 1493, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 10619.876333978027, "decode_payload_ms": 0.0, "receiver_prefill_ms": 107.23908396903425, "ttft_ms": 19.994916045106947, "decode_ms": 916.7372090741992}, "total_wall_ms": 1045.7014170242473, "ttft_ms": 19.994916045106947, "input_usage": {"receiver_prompt_tokens": 600, "sender_input_tokens": 2053, "sender_output_tokens": 512, "sender_prefill_ms": 348.635541042313, "sender_ttft_ms": 21.78091707173735, "sender_decode_ms": 10246.84870801866}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701105152, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6582726744, "mlx_cache_bytes": 3477464560, "process_rss_bytes": 6701105152}, "peak_prefill_memory": null, "peak_decode_memory": 6582726744, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['region'] != 'Sud' and row['units'] >= 2 and not row['returned'] and row['", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "a5360f49c84c5d88602af0cacb8eb14a906d47404d247d72e28a9c373feb8bd9", "output_hash": "ad6a28cb17112d61c6e0861769c09b6aca4ac8b11382ab844d0d76d38aeeeddd"}
83 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "dff5b8ff7f0b562672df6bac", "attempt_id": "4af52d4fb1cc", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:52.967207Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 1, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 165, "message_tokens": 67, "message_utf8_bytes": 238, "cache_tensor_bytes": null, "serialized_payload_bytes": 238, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2254.3724169954658, "decode_payload_ms": 0.0, "receiver_prefill_ms": 43.76500006765127, "ttft_ms": 20.686833071522415, "decode_ms": 18.83375004399568}, "total_wall_ms": 84.36074992641807, "ttft_ms": 20.686833071522415, "input_usage": {"receiver_prompt_tokens": 165, "sender_input_tokens": 2081, "sender_output_tokens": 85, "sender_prefill_ms": 492.3774159979075, "sender_ttft_ms": 25.297958054579794, "sender_decode_ms": 1734.3169170198962}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6701236224, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6314550016, "mlx_cache_bytes": 3624036480, "process_rss_bytes": 6701236224}, "peak_prefill_memory": null, "peak_decode_memory": 6314550016, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 1, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:NameError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c42d3913024ae8df3e3c52d4d39f2ffe452760aa6268f9e8a4cd162fdc0c1244", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
84 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c11304744694f15ee8ad5b33", "attempt_id": "072e89355803", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:55.240687Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 1}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 214, "message_tokens": 130, "message_utf8_bytes": 103, "cache_tensor_bytes": null, "serialized_payload_bytes": 103, "side_information_bytes": 0, "fetched_bytes": 269, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2144.001582986675, "decode_payload_ms": 0.0, "receiver_prefill_ms": 49.391667009331286, "ttft_ms": 20.21337498445064, "decode_ms": 926.9428750267252}, "total_wall_ms": 997.4543750286102, "ttft_ms": 20.21337498445064, "input_usage": {"receiver_prompt_tokens": 214, "sender_input_tokens": 2081, "sender_output_tokens": 85, "sender_prefill_ms": 341.33658302016556, "sender_ttft_ms": 22.545540938153863, "sender_decode_ms": 1777.7792499400675}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701236224, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6353527748, "mlx_cache_bytes": 3800132564, "process_rss_bytes": 6701236224}, "peak_prefill_memory": null, "peak_decode_memory": 6353527748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n price = row['unit_price'] * (1 - row['discount_pct'] / 100)\n if row['returned", "evidence_artifact": {"lookups": 1, "fetched_bytes": 269, "retrieved_refs": 1}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1ca01c6841ec89d2df6a744602238891e2a504aadf6d34de0cd09f8e803fe6dc", "output_hash": "1866ddef1765909117151568b3585703a3a42efe40e7c57f45cb199860548258"}
85 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "56ea44a15f71c558c2dced6e", "attempt_id": "caaced348dec", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:56.283336Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 5.879160636758321}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 2020, "message_tokens": 1937, "message_utf8_bytes": 8125, "cache_tensor_bytes": null, "serialized_payload_bytes": 1382, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.09729096200317144, "decode_payload_ms": 0.018749968148767948, "receiver_prefill_ms": 294.89733395166695, "ttft_ms": 14.12741607055068, "decode_ms": 1023.5364160034806}, "total_wall_ms": 1336.8795829592273, "ttft_ms": 14.12741607055068, "input_usage": {"receiver_prompt_tokens": 2020}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6701236224, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7052050480, "mlx_cache_bytes": 3800132564, "process_rss_bytes": 6701236224}, "peak_prefill_memory": null, "peak_decode_memory": 7052050480, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
86 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "da82041b0960ddc3a14c4229", "attempt_id": "6a38b839a652", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:58.001090Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6320513032}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 72843264, "serialized_payload_bytes": 72859340, "side_information_bytes": 16076, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 333.8079579407349, "doc_prefill_ms": 309.61087497416884, "compress_ms": 2.533124992623925, "serialize_ms": 17.797374981455505, "load_ms": 2.209541038610041, "receiver_prefill_ms": 0.0, "query_prefill_ms": 49.67045795638114, "ttft_ms": 49.67045795638114, "decode_ms": 1017.886417103}, "total_wall_ms": 1071.5985000133514, "ttft_ms": 49.67045795638114, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6847250432, "backend_memory": {"mlx_active_bytes": 6397190152, "mlx_peak_bytes": 6449848672, "mlx_cache_bytes": 4718079109, "process_rss_bytes": 6847250432}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6449848672, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "698a195b26016f12bd71d6631fda0686907c1fdcb84c16df4296b26693b09e5c", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
87 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "bbd615e60e849d8b1a7ba1ac", "attempt_id": "46733e8c5df8", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:27:59.142860Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6320513032}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "08df6c56032443d5", "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 72843264, "serialized_payload_bytes": 72859368, "side_information_bytes": 16104, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 24.76350008510053, "doc_prefill_ms": 309.61087497416884, "compress_ms": 2.834332990460098, "serialize_ms": 21.69116702862084, "load_ms": 12.23729201592505, "receiver_prefill_ms": 0.0, "query_prefill_ms": 40.432416019029915, "ttft_ms": 40.432416019029915, "decode_ms": 1000.5399170331657}, "total_wall_ms": 1054.9829159863293, "ttft_ms": 40.432416019029915, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6920404992, "backend_memory": {"mlx_active_bytes": 6397190152, "mlx_peak_bytes": 6449848560, "mlx_cache_bytes": 4718079109, "process_rss_bytes": 6920404992}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6449848560, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['region'] == 'Sud':\n continue\n if r['units", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2c13aad81bcd8dfdb14c874ddc3a115036f4e8d9db3242e3aa526e85a72a8286", "output_hash": "a4b5ee1863268030f5b49a1b3ed1224905e65c6b49f3a7ab91a764f8de64d358"}
88 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "d0d767d6b99935891bed0b25", "attempt_id": "cf27b3f98bac", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:00.269067Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6286408072}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 38697984, "serialized_payload_bytes": 38732174, "side_information_bytes": 34190, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 26.633040979504585, "doc_prefill_ms": 309.61087497416884, "compress_ms": 13.843290973454714, "serialize_ms": 12.345083057880402, "load_ms": 2.3097090888768435, "receiver_prefill_ms": 0.0, "query_prefill_ms": 24.407291086390615, "ttft_ms": 24.407291086390615, "decode_ms": 1126.3775830157101}, "total_wall_ms": 1154.8897090833634, "ttft_ms": 24.407291086390615, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6958333952, "backend_memory": {"mlx_active_bytes": 6330349960, "mlx_peak_bytes": 6395258492, "mlx_cache_bytes": 5054269957, "process_rss_bytes": 6958333952}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6395258492, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud':\n continue\n if r['units'] < 2:\n continue\n ", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "86fd69e1ab032c32583a0b7f11de3578bace1807d2de39b464d21bd2715100fa", "output_hash": "c03c392e692f4c667997049602f7e27418e41f00feeb390a815250c0569c1b13"}
89 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "b0e0b0cdf8a9e8bf185f2ee9", "attempt_id": "e3d8c7bde859", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:01.493320Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6268525576}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 20487168, "serialized_payload_bytes": 20521220, "side_information_bytes": 34052, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22.63975003734231, "doc_prefill_ms": 309.61087497416884, "compress_ms": 13.28904100228101, "serialize_ms": 8.920374908484519, "load_ms": 2.213416970334947, "receiver_prefill_ms": 0.0, "query_prefill_ms": 33.245792030356824, "ttft_ms": 33.245792030356824, "decode_ms": 1108.6429170100018}, "total_wall_ms": 1146.063124993816, "ttft_ms": 33.245792030356824, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6960431104, "backend_memory": {"mlx_active_bytes": 6294297608, "mlx_peak_bytes": 6354684588, "mlx_cache_bytes": 5121994885, "process_rss_bytes": 6960431104}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6354684588, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "user\nunknown\n the the the\n document\n document the\n the\n\n the The existing\n the\n\n\n the The\n\n\n the the the\n the the the, the the the the The the the The\n The the", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:IndentationError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "ca347fd0215673b4a988d4223cdd6c3b690c3e1c9eb09e181f013f0778625d27", "output_hash": "c02cea32658f86423a17c812c900737ca47ebf057e88e9371d3fa6d2eff848a6"}
90 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "90260663756f15d8ab415e67", "attempt_id": "682b8fe93539", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:02.701867Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1976, "keep": 494, "compress_ms": 7.7032080153003335}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226732, "side_information_bytes": 15916, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 14.727665926329792, "doc_prefill_ms": 309.61087497416884, "compress_ms": 7.759916945360601, "serialize_ms": 6.733249989338219, "load_ms": 0.9612080175429583, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.56879199761897, "ttft_ms": 32.56879199761897, "decode_ms": 855.7253750041127}, "total_wall_ms": 891.3752499502152, "ttft_ms": 32.56879199761897, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 44}, "provider_cached_usage": null, "process_memory": 6964920320, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5153287301, "process_rss_bytes": 6964920320}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for row in rows:\n if row['status'] == 'completed':\n total_revenue += row['amount']\n return total_revenue", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "77876ae624b46c34b40953b0b294518e8694fcb2fbdff1665ed3871727ddf56f", "output_hash": "c1a9a621f1cfa430e5d57319c441f0b3699df1bdd0916d47d16bd8d3b636329d"}
91 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "faf3c5252bcbb25b131280a7", "attempt_id": "58c23501aa9f", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:03.653051Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1976, "keep": 494, "compress_ms": 6.551417056471109}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226726, "side_information_bytes": 15910, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 12.743625091388822, "doc_prefill_ms": 309.61087497416884, "compress_ms": 6.587499985471368, "serialize_ms": 5.920208059251308, "load_ms": 0.679041026160121, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.208042076788843, "ttft_ms": 29.208042076788843, "decode_ms": 932.1196250384673}, "total_wall_ms": 963.8700410723686, "ttft_ms": 29.208042076788843, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6979092480, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5153287301, "process_rss_bytes": 6979092480}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0\n for row in rows:\n if row['region'] != 'S' and row['region'] != 'Centre':\n total += row['units'] * row['price", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "4dbdad308845b08c8ddd8b1fd3d33a76db9d3416f55bc59b4a5b91ea60ddf302", "output_hash": "1b45a1a25e468a6175f2069e3767b3b695f311b3606ae8a8aa1c17d84cd9c6a7"}
92 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "5949397e740909d42f727519", "attempt_id": "06844a495723", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:04.665719Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1976, "keep": 494, "window_tokens": 44, "compress_ms": 58.248165994882584}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226772, "side_information_bytes": 15956, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 65.47891697846353, "doc_prefill_ms": 309.61087497416884, "compress_ms": 60.018207994289696, "serialize_ms": 5.194208002649248, "load_ms": 0.6166669772937894, "receiver_prefill_ms": 0.0, "query_prefill_ms": 23.563917027786374, "ttft_ms": 23.563917027786374, "decode_ms": 943.9251670846716}, "total_wall_ms": 1035.5242089135572, "ttft_ms": 23.563917027786374, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6979223552, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5256199293, "process_rss_bytes": 6979223552}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Shud' or len(r['units']) < 2:\n continue\n if r['", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c4eae48b1e15863a34266e4931e222e4c99ef00239336c8a4747c8ecf10da1ad", "output_hash": "eee59974af47160590a101092589ed6092783b0739bc4edfcc774b7d9be5d4cc"}
93 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "4fc13be8620c3c81f55b6e3c", "attempt_id": "838ba2ff669e", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:05.814399Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6266249224}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1976, "keep": 494, "window_tokens": 19, "compress_ms": 58.98000008892268}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 18210816, "serialized_payload_bytes": 18226771, "side_information_bytes": 15955, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 65.83225005306304, "doc_prefill_ms": 309.61087497416884, "compress_ms": 59.280334040522575, "serialize_ms": 6.3120829872787, "load_ms": 0.8338750340044498, "receiver_prefill_ms": 0.0, "query_prefill_ms": 29.883750015869737, "ttft_ms": 29.883750015869737, "decode_ms": 947.136375005357}, "total_wall_ms": 979.7340830555186, "ttft_ms": 29.883750015869737, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6979289088, "backend_memory": {"mlx_active_bytes": 6288662536, "mlx_peak_bytes": 6329884912, "mlx_cache_bytes": 5266635901, "process_rss_bytes": 6979289088}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6329884912, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n rows = [r for r in rows if r['units'] >= 2 and r['region'] != 'S']\n rows = [r for r in rows if r['shipped_days']", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "c2c7ecc64c9caf7ce1f6ad2de4912a25a929a2366bc0119fd9f498ffff62858a", "output_hash": "a18179203926d78856727d2c151d583a3bfd58bcc61dc42e63a72dbf44494cba"}
94 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c46cf3e234e112a0f5f8112b", "attempt_id": "41b243f2a98e", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:06.840622Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6247374856}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 72.07979203667492, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (44,974) and (1,16,44,949) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
95 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2ec276c68d8afd8d728c36d2", "attempt_id": "99963ae12b8e", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:06.959128Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "932bac511f2c5ee3faf8c1f37b853fcd2be373db29c6a84b2d7f8c3ab71ee6f9", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6267434784}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1976, "keep": 988, "compress_ms": 8.043541922233999, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1937, "source_tokens_receiver": 1937, "query_tokens": 44, "message_tokens": 1976, "message_utf8_bytes": 8125, "cache_tensor_bytes": 19348992, "serialized_payload_bytes": 19382643, "side_information_bytes": 33651, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 25.42837499640882, "doc_prefill_ms": 309.61087497416884, "compress_ms": 16.244541970081627, "serialize_ms": 8.818165981210768, "load_ms": 1.8096249550580978, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.5376669857651, "ttft_ms": 30.5376669857651, "decode_ms": 1107.7149999327958}, "total_wall_ms": 1142.3345000948757, "ttft_ms": 30.5376669857651, "input_usage": {"receiver_prefix_tokens": 1976, "dense_cache_bytes": 72843264, "query_tokens": 44}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6999080960, "backend_memory": {"mlx_active_bytes": 6290392416, "mlx_peak_bytes": 6345574196, "mlx_cache_bytes": 5288831909, "process_rss_bytes": 6999080960}, "peak_prefill_memory": 7034568616, "peak_decode_memory": 6345574196, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n d = r['discount", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "6cd4d317f609771874f10328ee8c9d0593582762cc86c71221870cbe34b03bf2", "output_hash": "11ef43e7a25672ac1b5bda0cb1f1c298234481cd58e721fb63925aba7d5c84b9"}
96 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ada2dd091bdea89a3739786c", "attempt_id": "19610da4e3c9", "run_group_id": "E0_smoke:A0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:08.198628Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 60, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.01325004268437624, "decode_payload_ms": 0.0, "receiver_prefill_ms": 22.8534999769181, "ttft_ms": 19.033625023439527, "decode_ms": 18.955833977088332}, "total_wall_ms": 61.44804204814136, "ttft_ms": 19.033625023439527, "input_usage": {"receiver_prompt_tokens": 60}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6224191848, "mlx_cache_bytes": 131829180, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6224191848, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
97 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "3cd7ce88f7932e2fe3a5dca6", "attempt_id": "ff9af49a194b", "run_group_id": "E0_smoke:A1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:08.286310Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 3314, "message_tokens": 3244, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 15217, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011375057511031628, "decode_payload_ms": 0.0, "receiver_prefill_ms": 623.0147909373045, "ttft_ms": 22.94987498316914, "decode_ms": 757.846082909964}, "total_wall_ms": 1410.354790976271, "ttft_ms": 22.94987498316914, "input_usage": {"receiver_prompt_tokens": 3314}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054791664, "mlx_cache_bytes": 1964671360, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 7054791664, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
98 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e9ea83259bd11521d23b5992", "attempt_id": "cda8cd210413", "run_group_id": "E0_smoke:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:09.724185Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 3244}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 591, "message_tokens": 516, "message_utf8_bytes": 2729, "cache_tensor_bytes": null, "serialized_payload_bytes": 2729, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 3.2423330703750253, "decode_payload_ms": 0.0, "receiver_prefill_ms": 101.38687491416931, "ttft_ms": 27.95499993953854, "decode_ms": 100.74745898600668}, "total_wall_ms": 231.90870904363692, "ttft_ms": 27.95499993953854, "input_usage": {"receiver_prompt_tokens": 591}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6564311128, "mlx_cache_bytes": 2357264768, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6564311128, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD pooling approach", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "11ac585095e23aad2dc25c5d250ea6fc439122cd288e6814f52c1cea040db97b", "output_hash": "b396b6ce678924171263f2c16423a6fb5ea85d772a5b7e43ad73b71e42b11ad7"}
99 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "2c10d09f655cb2217d462ef3", "attempt_id": "fb2521d6b85c", "run_group_id": "E0_smoke:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:09.990150Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 500, "n_sentences": 11}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 571, "message_tokens": 500, "message_utf8_bytes": 2058, "cache_tensor_bytes": null, "serialized_payload_bytes": 2058, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 4.259041976183653, "decode_payload_ms": 0.0, "receiver_prefill_ms": 102.35675005242229, "ttft_ms": 26.3324580155313, "decode_ms": 24.861750076524913}, "total_wall_ms": 155.5670410161838, "ttft_ms": 26.3324580155313, "input_usage": {"receiver_prompt_tokens": 571}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6552219736, "mlx_cache_bytes": 2704474496, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6552219736, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "a804cda2443cf12a08e11e226234585e0d1edf60d020061934488a33516d413b", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
100 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "e5a31efff5f91b7eb69a8bc1", "attempt_id": "d3c27d3f5d40", "run_group_id": "E0_smoke:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:10.174696Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 506, "n_sentences": 23}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 578, "message_tokens": 507, "message_utf8_bytes": 2495, "cache_tensor_bytes": null, "serialized_payload_bytes": 2495, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.6516250325366855, "decode_payload_ms": 0.0, "receiver_prefill_ms": 122.30041704606265, "ttft_ms": 17.13062508497387, "decode_ms": 417.9142080247402}, "total_wall_ms": 560.8353751013055, "ttft_ms": 17.13062508497387, "input_usage": {"receiver_prompt_tokens": 578}, "output_usage": {"output_tokens": 19}, "provider_cached_usage": null, "process_memory": 6773866496, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6557036632, "mlx_cache_bytes": 3041952128, "process_rss_bytes": 6773866496}, "peak_prefill_memory": null, "peak_decode_memory": 6557036632, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension to VLAD where hard assignment is replaced with soft assignment for end-to-end training.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.27027027027027023, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "785023803d7f2c38c80debcdc8309ffa1c94e3c99227c52a36e7e66d830d2d9e", "output_hash": "e746ed4fb6926b79617f224cc269a459de798a399cfc39ed7c9b314e07455f73"}
101 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "7ee46ca5b5986914ad0f70a5", "attempt_id": "27e4e6a2dd08", "run_group_id": "E0_smoke:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:16.132554Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 294, "message_tokens": 220, "message_utf8_bytes": 1044, "cache_tensor_bytes": null, "serialized_payload_bytes": 1044, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 5371.418667025864, "decode_payload_ms": 0.0, "receiver_prefill_ms": 71.67716696858406, "ttft_ms": 19.78166599292308, "decode_ms": 70.39725000504404}, "total_wall_ms": 163.06179203093052, "ttft_ms": 19.78166599292308, "input_usage": {"receiver_prompt_tokens": 294, "sender_input_tokens": 3360, "sender_output_tokens": 220, "sender_prefill_ms": 656.0712079517543, "sender_ttft_ms": 15.746999997645617, "sender_decode_ms": 4695.432165986858}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6773899264, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6424421636, "mlx_cache_bytes": 3801168528, "process_rss_bytes": 6773899264}, "peak_prefill_memory": null, "peak_decode_memory": 6424421636, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3b08a9dd9882d778138853f9b9a6a55c7176ad67e2db289645e13b8fc1bac590", "output_hash": "4a81758367f5a2ca4e9aa1c032ddd94e54a15042d670c320c4955ecf300ae450"}
102 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "fa7aaf225eb9ea8ec9a07212", "attempt_id": "e68901764e49", "run_group_id": "E0_smoke:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:24.829477Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 5, "unresolved_quotes": 1, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 306, "message_tokens": 221, "message_utf8_bytes": 756, "cache_tensor_bytes": null, "serialized_payload_bytes": 756, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 8506.694042007439, "decode_payload_ms": 0.0, "receiver_prefill_ms": 55.022082990035415, "ttft_ms": 21.521249902434647, "decode_ms": 58.01579204853624}, "total_wall_ms": 135.69395896047354, "ttft_ms": 21.521249902434647, "input_usage": {"receiver_prompt_tokens": 306, "sender_input_tokens": 3388, "sender_output_tokens": 380, "sender_prefill_ms": 809.7484159516171, "sender_ttft_ms": 25.65774996764958, "sender_decode_ms": 7667.614832986146}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 6773932032, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6434170164, "mlx_cache_bytes": 4717179968, "process_rss_bytes": 6773932032}, "peak_prefill_memory": null, "peak_decode_memory": 6434170164, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "pooling method", "evidence_artifact": {"n_facts": 5, "unresolved_quotes": 1, "parse_ok": true}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "3bb6e01e5b80c84d0441a0b53d50795f440126a2d273b098a4adc01abc60175f", "output_hash": "8c7e3b1576d203644fb74ba50478132796f7da6038f8971317fa262590fd6536"}
103 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "fc7f1dec81af51ed135642f5", "attempt_id": "7ca101f1952a", "run_group_id": "E0_smoke:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:33.574996Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 544, "message_tokens": 473, "message_utf8_bytes": 371, "cache_tensor_bytes": null, "serialized_payload_bytes": 371, "side_information_bytes": 0, "fetched_bytes": 1373, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 8585.026083979756, "decode_payload_ms": 0.0, "receiver_prefill_ms": 104.83375005424023, "ttft_ms": 22.171457996591926, "decode_ms": 62.47520900797099}, "total_wall_ms": 190.89829095173627, "ttft_ms": 22.171457996591926, "input_usage": {"receiver_prompt_tokens": 544, "sender_input_tokens": 3388, "sender_output_tokens": 380, "sender_prefill_ms": 719.3035829113796, "sender_ttft_ms": 25.42624995112419, "sender_decode_ms": 7836.96520794183}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 6773915648, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 6535016536, "mlx_cache_bytes": 5052789824, "process_rss_bytes": 6773915648}, "peak_prefill_memory": null, "peak_decode_memory": 6535016536, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "pooling method", "evidence_artifact": {"lookups": 4, "fetched_bytes": 1373, "retrieved_refs": 4}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "15bd5c2c7fa5c1256d4044abb684ec5f8ec847c3e301d4402dc3bc463138b521", "output_hash": "8c7e3b1576d203644fb74ba50478132796f7da6038f8971317fa262590fd6536"}
104 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "9c0b36e54acecbc04d506a33", "attempt_id": "79213f5e6476", "run_group_id": "E0_smoke:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:33.790608Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6171877384}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 2.8798258894776683}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 3314, "message_tokens": 3244, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 5284, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.09345798753201962, "decode_payload_ms": 0.02058397512882948, "receiver_prefill_ms": 704.8352500423789, "ttft_ms": 24.021124932914972, "decode_ms": 774.4359999196604}, "total_wall_ms": 1509.2177080223337, "ttft_ms": 24.021124932914972, "input_usage": {"receiver_prompt_tokens": 3314}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 6773915648, "backend_memory": {"mlx_active_bytes": 6171877384, "mlx_peak_bytes": 7054791664, "mlx_cache_bytes": 5052789824, "process_rss_bytes": 6773915648}, "peak_prefill_memory": null, "peak_decode_memory": 7054791664, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
105 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ee0a4bf8945963b7984eda04", "attempt_id": "7a5edbe6a2aa", "run_group_id": "E0_smoke:B0", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:35.959615Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6416064520}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 121024512, "serialized_payload_bytes": 121040621, "side_information_bytes": 16109, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 635.4355829535052, "doc_prefill_ms": 601.0837500216439, "compress_ms": 4.678208031691611, "serialize_ms": 23.705999949015677, "load_ms": 2.692666952498257, "receiver_prefill_ms": 0.0, "query_prefill_ms": 62.39358300808817, "ttft_ms": 62.39358300808817, "decode_ms": 771.4831250486895}, "total_wall_ms": 839.4441249547526, "ttft_ms": 62.39358300808817, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 7016480768, "backend_memory": {"mlx_active_bytes": 6539927560, "mlx_peak_bytes": 6590084908, "mlx_cache_bytes": 5976129213, "process_rss_bytes": 7016480768}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6590084908, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e72b7ab2934cfcf6888dcc2d829003003dc376d34928371486fae694c1a2098c", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
106 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "6d448ed450568948fe73b33b", "attempt_id": "962989a845d3", "run_group_id": "E0_smoke:B1", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:36.848715Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6416064520}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "35aaed2f1f78202e", "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 121024512, "serialized_payload_bytes": 121040649, "side_information_bytes": 16137, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 25.53833299316466, "doc_prefill_ms": 601.0837500216439, "compress_ms": 1.076459069736302, "serialize_ms": 24.271624977700412, "load_ms": 16.446624998934567, "receiver_prefill_ms": 0.0, "query_prefill_ms": 49.789458978921175, "ttft_ms": 49.789458978921175, "decode_ms": 747.1429159631953}, "total_wall_ms": 816.2833329988644, "ttft_ms": 49.789458978921175, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 34}, "provider_cached_usage": null, "process_memory": 7137984512, "backend_memory": {"mlx_active_bytes": 6539927560, "mlx_peak_bytes": 6590084796, "mlx_cache_bytes": 5976129213, "process_rss_bytes": 7137984512}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6590084796, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to map noisy or irrelevant content into ghost clusters, which are not included during feature aggregation.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "e188c41f1d9a05bc6e14927399c189fe21e3a4e72cabf24975d723a2d32d953f", "output_hash": "ba5fb8f0ad59cb232cd6f80c40ee8d80776fb7390bc885bdbaae680052882740"}
107 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "1ac34c36ff3d7315fc796eae", "attempt_id": "d79add9587e3", "run_group_id": "E0_smoke:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:37.723313Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6360621064}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 64294272, "serialized_payload_bytes": 64328508, "side_information_bytes": 34236, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 33.56612497009337, "doc_prefill_ms": 601.0837500216439, "compress_ms": 16.29454200156033, "serialize_ms": 16.862374963238835, "load_ms": 3.4387909108772874, "receiver_prefill_ms": 0.0, "query_prefill_ms": 132.63883395120502, "ttft_ms": 132.63883395120502, "decode_ms": 75.75041695963591}, "total_wall_ms": 214.81683303136379, "ttft_ms": 132.63883395120502, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7201554432, "backend_memory": {"mlx_active_bytes": 6426681352, "mlx_peak_bytes": 6493745728, "mlx_cache_bytes": 6267740861, "process_rss_bytes": 7201554432}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6493745728, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "2a1a43b3fa4a399c34ba6ec7a1917e4c670fbe4963cb057ca7ec0551e351b23d", "output_hash": "75a872242c44bab0506befcac9b6a38c36a90ed87a8b8fbe27f7514cdf9dbb77"}
108 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "ca17bd88ada65a286ee0a6f7", "attempt_id": "3187f01cea42", "run_group_id": "E0_smoke:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:37.984179Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6329950216}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 34038144, "serialized_payload_bytes": 34072304, "side_information_bytes": 34160, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22.94916706159711, "doc_prefill_ms": 601.0837500216439, "compress_ms": 11.391667067073286, "serialize_ms": 11.177083011716604, "load_ms": 2.3093329509720206, "receiver_prefill_ms": 0.0, "query_prefill_ms": 143.6609170632437, "ttft_ms": 143.6609170632437, "decode_ms": 1126.990083954297}, "total_wall_ms": 1278.614499955438, "ttft_ms": 143.6609170632437, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7232307200, "backend_memory": {"mlx_active_bytes": 6366519304, "mlx_peak_bytes": 6424664597, "mlx_cache_bytes": 6436792893, "process_rss_bytes": 7232307200}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6424664597, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": " answer is is is the map is is the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the theuss the the the the", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "1b95c993af3d21a934574f795df3e1fbf79b282247215a9eb48cbf2a848afa55", "output_hash": "2c0f9e0ae3f39ad1661d054a003adda4563e4914e846c6b20e1739fe08202a49"}
109 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "0b133b77ac8a96e5c6b5bc99", "attempt_id": "72a86d76f83b", "run_group_id": "E0_smoke:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:39.307416Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6325231624}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 3283, "keep": 821, "compress_ms": 8.413915988057852}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281293, "side_information_bytes": 15949, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19.421624951064587, "doc_prefill_ms": 601.0837500216439, "compress_ms": 8.4537910297513, "serialize_ms": 10.77312498819083, "load_ms": 1.2885419419035316, "receiver_prefill_ms": 0.0, "query_prefill_ms": 30.25900002103299, "ttft_ms": 30.25900002103299, "decode_ms": 44.62770896498114}, "total_wall_ms": 79.53341701067984, "ttft_ms": 30.25900002103299, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 7236452352, "backend_memory": {"mlx_active_bytes": 6357082120, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6446230077, "process_rss_bytes": 7236452352}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "203c88371d6551640470222a8f0db73f9cca2629b2d98351a486b0cc01f024f0", "output_hash": "aea88d526d067608708536c90844a71e2fdd73002205449a86f1441074ce02df"}
110 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "cecbdd65d043b664a2811af6", "attempt_id": "86745d804a03", "run_group_id": "E0_smoke:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:39.430851Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6325231624}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 3283, "keep": 821, "compress_ms": 7.425334071740508}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281288, "side_information_bytes": 15944, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 16.988249961286783, "doc_prefill_ms": 601.0837500216439, "compress_ms": 7.470625103451312, "serialize_ms": 9.28033294621855, "load_ms": 0.7437919266521931, "receiver_prefill_ms": 0.0, "query_prefill_ms": 32.686542021110654, "ttft_ms": 32.686542021110654, "decode_ms": 219.86454201396555}, "total_wall_ms": 256.39504101127386, "ttft_ms": 32.686542021110654, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 11}, "provider_cached_usage": null, "process_memory": 7259455488, "backend_memory": {"mlx_active_bytes": 6357082120, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6446230077, "process_rss_bytes": 7259455488}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of VLAD with additional \"ghost\" clusters", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.588235294117647, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "70ea332320edce996b98525a13cea25477ebb6008fd834fc605091509e59c638", "output_hash": "2186a0d38b03b30a9f490d89c4029f93aed3afbdfe49ce3b728692172ec75684"}
111 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "34c653518a2822f0248089b7", "attempt_id": "f3b6dad3ff16", "run_group_id": "E0_smoke:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:39.712620Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6294560776}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3283, "keep": 821, "window_tokens": 31, "compress_ms": 73.18974996451288}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281333, "side_information_bytes": 15989, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 83.91400007531047, "doc_prefill_ms": 601.0837500216439, "compress_ms": 76.17241598200053, "serialize_ms": 7.476332946680486, "load_ms": 0.6962079787626863, "receiver_prefill_ms": 0.0, "query_prefill_ms": 23.969750036485493, "ttft_ms": 23.969750036485493, "decode_ms": 469.9488329933956}, "total_wall_ms": 581.5955420257524, "ttft_ms": 23.969750036485493, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 23}, "provider_cached_usage": null, "process_memory": 7261257728, "backend_memory": {"mlx_active_bytes": 6358261768, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6516845253, "process_rss_bytes": 7261257728}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "GhostVLAD is an extension of the NetVLAD approach, adding Ghost clusters to improve aggregation of local features.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "47e80056bbc1a4eb0bd50907f4df60fa9febc185c53aa9bfb01a998dafb1fe62", "output_hash": "fecf4b3747739da00410fc6decc4ba4dde1d66f25ed88cb4a76dc7d3b1f7a46b"}
112 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "963b06874aa12c25afdf0c63", "attempt_id": "6b3a3d3bfe73", "run_group_id": "E0_smoke:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:40.399242Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6325231624}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3283, "keep": 821, "window_tokens": 19, "compress_ms": 73.75320896971971}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 30265344, "serialized_payload_bytes": 30281333, "side_information_bytes": 15989, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.70887504052371, "doc_prefill_ms": 601.0837500216439, "compress_ms": 73.89700005296618, "serialize_ms": 5.637041991576552, "load_ms": 1.3112499145790935, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.88529098406434, "ttft_ms": 31.88529098406434, "decode_ms": 456.10237505752593}, "total_wall_ms": 492.16875003185123, "ttft_ms": 31.88529098406434, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 22}, "provider_cached_usage": null, "process_memory": 7269089280, "backend_memory": {"mlx_active_bytes": 6358261768, "mlx_peak_bytes": 6390658748, "mlx_cache_bytes": 6539504325, "process_rss_bytes": 7269089280}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6390658748, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD approach is a pooling method used for language identification, inspired by recent work on speaker identification.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "47e80056bbc1a4eb0bd50907f4df60fa9febc185c53aa9bfb01a998dafb1fe62", "output_hash": "ba29f94ed96dd49c401ef0be023381ab5ee8b988ed76f46cdc19c5602ac9b29c"}
113 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "f7d0a3fc2321b3c9704b89ec", "attempt_id": "d2a08aee8841", "run_group_id": "E0_smoke:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:40.919206Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6294560776}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware"}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": null, "message_tokens": null, "message_utf8_bytes": null, "cache_tensor_bytes": null, "serialized_payload_bytes": null, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {}, "total_wall_ms": 87.07641693763435, "ttft_ms": null, "input_usage": {}, "output_usage": {}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": null, "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": null, "status": "implementation_error", "error_code": "ValueError: [broadcast_shapes] Shapes (31,1574) and (1,16,31,1533) cannot be broadcast.", "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": null, "output_hash": null}
114 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke", "trial_id": "c08a83e61897f15f24d32a73", "attempt_id": "a9fbb7a566b2", "run_group_id": "E0_smoke:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "966dc0b4711f65959caf629a7f306c5833596a1dea7d7e43c07a6279909beec3", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:28:41.064575Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"sender": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b", "receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6327222532}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 3283, "keep": 1642, "compress_ms": 8.948500035330653, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3244, "source_tokens_receiver": 3244, "query_tokens": 31, "message_tokens": 3283, "message_utf8_bytes": 15217, "cache_tensor_bytes": 32156928, "serialized_payload_bytes": 32191192, "side_information_bytes": 34264, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 33.74208300374448, "doc_prefill_ms": 601.0837500216439, "compress_ms": 20.669832942076027, "serialize_ms": 12.674166937358677, "load_ms": 2.0706249633803964, "receiver_prefill_ms": 0.0, "query_prefill_ms": 23.13250000588596, "ttft_ms": 23.13250000588596, "decode_ms": 386.1410419922322}, "total_wall_ms": 414.49383401777595, "ttft_ms": 23.13250000588596, "input_usage": {"receiver_prefix_tokens": 3283, "dense_cache_bytes": 121024512, "query_tokens": 31}, "output_usage": {"output_tokens": 17}, "provider_cached_usage": null, "process_memory": 7299727360, "backend_memory": {"mlx_active_bytes": 6361179844, "mlx_peak_bytes": 6420485060, "mlx_cache_bytes": 6590475617, "process_rss_bytes": 7299727360}, "peak_prefill_memory": 7054791664, "peak_decode_memory": 6420485060, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "A pooling technique for aggregating frame-level features into a single utterance-level feature.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke.log", "payload_hash": "6f0cea2ea0fe134b9e5552bf881dec82625be5d2070aeab1427d56c909191a1b", "output_hash": "3ffdac3b266bbffe84697d9ce063315e72265a67e2c7d367e858dea2fc84ec04"}
added runs/raw/E0_smoke_llama.jsonl +114 −0
@@ -0,0 +1,114 @@
1 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "1acce7293fdf27d3784c6e81", "attempt_id": "3a91072adb5b", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:10.997570Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425501000}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 86, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 104.80358300264925, "ttft_ms": 38.55491697322577, "decode_ms": 42.36904205754399}, "total_wall_ms": 194.91720804944634, "ttft_ms": 38.55491697322577, "input_usage": {"receiver_prompt_tokens": 86}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6927237120, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6512662392, "mlx_cache_bytes": 88274304, "process_rss_bytes": 6927237120}, "peak_prefill_memory": null, "peak_decode_memory": 6512662392, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "b764cdc0eab7137467211272fa539f1260d1bf2e71bcf6ff3bdc960f5c16aa14"}
2 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7e57ab223f53fe54106b58be", "attempt_id": "783d0cb42803", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.222377Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 91, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 49.940416938625276, "ttft_ms": 32.91054104920477, "decode_ms": 55.94099999871105}, "total_wall_ms": 139.85866599250585, "ttft_ms": 32.91054104920477, "input_usage": {"receiver_prompt_tokens": 91}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6927417344, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6516209100, "mlx_cache_bytes": 151221992, "process_rss_bytes": 6927417344}, "peak_prefill_memory": null, "peak_decode_memory": 6516209100, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
3 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "80eba4737155699c0a7c69f8", "attempt_id": "ef9db7ee8bcb", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.386669Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 91, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 60.74287497904152, "ttft_ms": 34.07541697379202, "decode_ms": 90.18166700843722}, "total_wall_ms": 185.84875005763024, "ttft_ms": 34.07541697379202, "input_usage": {"receiver_prompt_tokens": 91}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6927433728, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6516209100, "mlx_cache_bytes": 151221992, "process_rss_bytes": 6927433728}, "peak_prefill_memory": null, "peak_decode_memory": 6516209100, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
4 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0f310404290cf683416256cc", "attempt_id": "55ce2bb3b2ca", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.596095Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 88, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.006209011189639568, "decode_payload_ms": 0.0, "receiver_prefill_ms": 53.37033304385841, "ttft_ms": 35.44983407482505, "decode_ms": 62.86750000435859}, "total_wall_ms": 152.5538330897689, "ttft_ms": 35.44983407482505, "input_usage": {"receiver_prompt_tokens": 88}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6927450112, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6519993804, "mlx_cache_bytes": 177665768, "process_rss_bytes": 6927450112}, "peak_prefill_memory": null, "peak_decode_memory": 6519993804, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
5 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "a06eb15f6c19a57624f42a03", "attempt_id": "8204848563f0", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:11.774406Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1862, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 444.0474580042064, "ttft_ms": 114.788957987912, "decode_ms": 270.48599999397993}, "total_wall_ms": 832.6459999661893, "ttft_ms": 114.788957987912, "input_usage": {"receiver_prompt_tokens": 1862}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6927777792, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7120900424, "mlx_cache_bytes": 884249596, "process_rss_bytes": 6927777792}, "peak_prefill_memory": null, "peak_decode_memory": 7120900424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
6 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b1d7b1cce8d3a973f5a29a1a", "attempt_id": "6bbc100b1afa", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:12.629456Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 495.7376670790836, "ttft_ms": 44.04479102231562, "decode_ms": 340.5430830316618}, "total_wall_ms": 884.0085410047323, "ttft_ms": 44.04479102231562, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6695288832, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 1354510628, "process_rss_bytes": 6695288832}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
7 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0d0757aa5d272f5e5f06f1f4", "attempt_id": "b5a52cd8b5e1", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:13.535016Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 451.73691597301513, "ttft_ms": 37.32558398041874, "decode_ms": 178.22666699066758}, "total_wall_ms": 670.3590000979602, "ttft_ms": 37.32558398041874, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6641074176, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 1354510628, "process_rss_bytes": 6641074176}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
8 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d29228b6041b9db070f72d7c", "attempt_id": "b70bcb139367", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:14.229625Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1864, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 7280, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.011915923096239567, "decode_payload_ms": 0.0, "receiver_prefill_ms": 440.22891600616276, "ttft_ms": 39.44133303593844, "decode_ms": 205.141541082412}, "total_wall_ms": 687.8513750853017, "ttft_ms": 39.44133303593844, "input_usage": {"receiver_prompt_tokens": 1864}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6551306240, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7159222632, "mlx_cache_bytes": 1594913060, "process_rss_bytes": 6551306240}, "peak_prefill_memory": null, "peak_decode_memory": 7159222632, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
9 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "4c3a0a4bad55488d3a6874e2", "attempt_id": "3170a08d5ad8", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:14.941063Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 617, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 177.61712498031557, "ttft_ms": 45.229541952721775, "decode_ms": 52.08924994803965}, "total_wall_ms": 276.7296659294516, "ttft_ms": 45.229541952721775, "input_usage": {"receiver_prompt_tokens": 617}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6554402816, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6867467768, "mlx_cache_bytes": 2050112196, "process_rss_bytes": 6554402816}, "peak_prefill_memory": null, "peak_decode_memory": 6867467768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
10 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d422269a403ca89ddb2d4079", "attempt_id": "0521e5fc82fd", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:15.236832Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 622, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 125.95404102467, "ttft_ms": 37.89616597350687, "decode_ms": 391.38012495823205}, "total_wall_ms": 556.9134170655161, "ttft_ms": 37.89616597350687, "input_usage": {"receiver_prompt_tokens": 622}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6554009600, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6878428684, "mlx_cache_bytes": 2363245688, "process_rss_bytes": 6554009600}, "peak_prefill_memory": null, "peak_decode_memory": 6878428684, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
11 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d1de1204b1dde86818bd5184", "attempt_id": "38fb4366138f", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:15.818841Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 622, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 168.85512496810406, "ttft_ms": 48.81137504708022, "decode_ms": 38.724916987121105}, "total_wall_ms": 258.002792019397, "ttft_ms": 48.81137504708022, "input_usage": {"receiver_prompt_tokens": 622}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6554058752, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6882262540, "mlx_cache_bytes": 2363245688, "process_rss_bytes": 6554058752}, "peak_prefill_memory": null, "peak_decode_memory": 6882262540, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
12 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "a97ce48716cccfffb0df7d46", "attempt_id": "8aff5e407615", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:16.096513Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1766}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 619, "message_tokens": 516, "message_utf8_bytes": 1940, "cache_tensor_bytes": null, "serialized_payload_bytes": 1940, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.5928338980302215, "decode_payload_ms": 0.0, "receiver_prefill_ms": 154.10387504380196, "ttft_ms": 35.48179194331169, "decode_ms": 60.69491698872298}, "total_wall_ms": 251.7090420005843, "ttft_ms": 35.48179194331169, "input_usage": {"receiver_prompt_tokens": 619}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6554173440, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6900022796, "mlx_cache_bytes": 2602517624, "process_rss_bytes": 6554173440}, "peak_prefill_memory": null, "peak_decode_memory": 6900022796, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "83c1970dac87ef7396a52d14a2764a080806f090cf3b37207beb09b20af11f42", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
13 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "65b2e0d942958d4955094a38", "attempt_id": "168a1109b45c", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:16.370902Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 606, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 174.26633299328387, "ttft_ms": 25.856332969851792, "decode_ms": 169.48837495874614}, "total_wall_ms": 371.1075000464916, "ttft_ms": 25.856332969851792, "input_usage": {"receiver_prompt_tokens": 606}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6556090368, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6861684216, "mlx_cache_bytes": 2940961912, "process_rss_bytes": 6556090368}, "peak_prefill_memory": null, "peak_decode_memory": 6861684216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Haruki Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "55a01495263476a6da84eabe2c22899f74b3909040b9d9d4fe8a8b6431ff9e18"}
14 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2a898ab433c5394686322580", "attempt_id": "2dbaac58758b", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:16.760564Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 611, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 140.41587500832975, "ttft_ms": 38.561749970540404, "decode_ms": 40.86958395782858}, "total_wall_ms": 221.20654198806733, "ttft_ms": 38.561749970540404, "input_usage": {"receiver_prompt_tokens": 611}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6556106752, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6864387576, "mlx_cache_bytes": 3282191480, "process_rss_bytes": 6556106752}, "peak_prefill_memory": null, "peak_decode_memory": 6864387576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
15 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ec7b2a9fe84bf0e283f64e2a", "attempt_id": "4c277500c5b6", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:17.001906Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 611, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 169.2981249652803, "ttft_ms": 23.593208054080606, "decode_ms": 54.50175004079938}, "total_wall_ms": 248.86029202025384, "ttft_ms": 23.593208054080606, "input_usage": {"receiver_prompt_tokens": 611}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6557040640, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6864387576, "mlx_cache_bytes": 3282191480, "process_rss_bytes": 6557040640}, "peak_prefill_memory": null, "peak_decode_memory": 6864387576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
16 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0c1493bd8cd834eeb7aa0785", "attempt_id": "9169bdcba916", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:17.272578Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 509, "n_sentences": 18}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 608, "message_tokens": 509, "message_utf8_bytes": 1536, "cache_tensor_bytes": null, "serialized_payload_bytes": 1536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.271499950438738, "decode_payload_ms": 0.0, "receiver_prefill_ms": 180.0412500742823, "ttft_ms": 43.045250000432134, "decode_ms": 190.88037498295307}, "total_wall_ms": 415.3167080366984, "ttft_ms": 43.045250000432134, "input_usage": {"receiver_prompt_tokens": 608}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6557106176, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6893698552, "mlx_cache_bytes": 3438888056, "process_rss_bytes": 6557106176}, "peak_prefill_memory": null, "peak_decode_memory": 6893698552, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ca1d3fed207a8962d88341b7fb1b07da00df9d8e507d786a2e99c06b18d0a62", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
17 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e9f693f84c29d8efb40e2291", "attempt_id": "79119d577b43", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:17.707911Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 511, "n_sentences": 28}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 608, "message_tokens": 511, "message_utf8_bytes": 1555, "cache_tensor_bytes": null, "serialized_payload_bytes": 1555, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.141499960795045, "decode_payload_ms": 0.0, "receiver_prefill_ms": 137.3269580071792, "ttft_ms": 42.00733406469226, "decode_ms": 276.81891701649874}, "total_wall_ms": 458.59983400441706, "ttft_ms": 42.00733406469226, "input_usage": {"receiver_prompt_tokens": 608}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6557253632, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6901202424, "mlx_cache_bytes": 3438888056, "process_rss_bytes": 6557253632}, "peak_prefill_memory": null, "peak_decode_memory": 6901202424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "0ff405ac03e5a10e14d572266d362d361b4219fa184f9be3bb806057879fd9ac", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
18 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "72b77475732181169f7e78f8", "attempt_id": "1910237ecabe", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:18.186661Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 508, "n_sentences": 29}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 610, "message_tokens": 508, "message_utf8_bytes": 1715, "cache_tensor_bytes": null, "serialized_payload_bytes": 1715, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.1243750341236591, "decode_payload_ms": 0.0, "receiver_prefill_ms": 195.8667910657823, "ttft_ms": 23.60216702800244, "decode_ms": 335.8849590877071}, "total_wall_ms": 557.7511669835076, "ttft_ms": 23.60216702800244, "input_usage": {"receiver_prompt_tokens": 610}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6557384704, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6864387576, "mlx_cache_bytes": 3438888056, "process_rss_bytes": 6557384704}, "peak_prefill_memory": null, "peak_decode_memory": 6864387576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2a8e20793db30181404c83877c2fa0ce0e9f4ce155842796b2e9a4db57e4d166", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
19 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3aa7131806a41f29a7e7635a", "attempt_id": "e3374140fba2", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:18.766348Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 512, "n_sentences": 29}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 614, "message_tokens": 512, "message_utf8_bytes": 1561, "cache_tensor_bytes": null, "serialized_payload_bytes": 1561, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.0653340723365545, "decode_payload_ms": 0.0, "receiver_prefill_ms": 152.72070898208767, "ttft_ms": 38.70024997740984, "decode_ms": 168.77012501936406}, "total_wall_ms": 362.51762497704476, "ttft_ms": 38.70024997740984, "input_usage": {"receiver_prompt_tokens": 614}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6557630464, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6897040888, "mlx_cache_bytes": 3597091960, "process_rss_bytes": 6557630464}, "peak_prefill_memory": null, "peak_decode_memory": 6897040888, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "a53881c8306c0d00c837f52a61f488dcff8e4df7502a72cc68c383b3156842b8", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
20 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2623ac986d4ae09b6f38cb6b", "attempt_id": "c118ee4ca756", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:19.148562Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 511, "n_sentences": 22}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 606, "message_tokens": 507, "message_utf8_bytes": 1893, "cache_tensor_bytes": null, "serialized_payload_bytes": 1893, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.0557500645518303, "decode_payload_ms": 0.0, "receiver_prefill_ms": 140.03133296500891, "ttft_ms": 41.58537497278303, "decode_ms": 201.92399993538857}, "total_wall_ms": 386.04374998249114, "ttft_ms": 41.58537497278303, "input_usage": {"receiver_prompt_tokens": 606}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6558023680, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6861684216, "mlx_cache_bytes": 3597091960, "process_rss_bytes": 6558023680}, "peak_prefill_memory": null, "peak_decode_memory": 6861684216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "22d6a96c1f0de138d69e1c361d6afdd8664290d33341c244479d5699bb0376d5", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
21 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c297d644a762a42131883b7d", "attempt_id": "bad721ed6bbf", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:36.584206Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 612, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 224.51616707257926, "ttft_ms": 51.759208901785314, "decode_ms": 58.00508300308138}, "total_wall_ms": 335.8493749983609, "ttft_ms": 51.759208901785314, "input_usage": {"receiver_prompt_tokens": 612, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6561136640, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6903856632, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561136640}, "peak_prefill_memory": null, "peak_decode_memory": 6903856632, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
22 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "af5ca197d146c72042e02b45", "attempt_id": "d46502500ded", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:36.941363Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 617, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 149.41387507133186, "ttft_ms": 42.637665988877416, "decode_ms": 313.3010419551283}, "total_wall_ms": 506.7510830704123, "ttft_ms": 42.637665988877416, "input_usage": {"receiver_prompt_tokens": 617, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6561169408, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6867467768, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561169408}, "peak_prefill_memory": null, "peak_decode_memory": 6867467768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 916-F, Fermont", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.28571428571428575}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "90f0dea38d92a987240fd60c09e83237873c207c6079538eed9a620045be71c5"}
23 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "55c02754131bfc6f507a1cf8", "attempt_id": "fe9605bbc93f", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:37.466533Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 617, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 199.5497919851914, "ttft_ms": 48.74649993143976, "decode_ms": 90.31466708984226}, "total_wall_ms": 339.81412497814745, "ttft_ms": 48.74649993143976, "input_usage": {"receiver_prompt_tokens": 617, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6561185792, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6867467768, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561185792}, "peak_prefill_memory": null, "peak_decode_memory": 6867467768, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "68.6281", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "dcf770b51da1816894e8b51a64ce9116a511525b5b251eab5e28faebddc3df9f"}
24 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7cd74952809d0bb6cd5c0200", "attempt_id": "b25eefb17888", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:37.825977Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 614, "message_tokens": 512, "message_utf8_bytes": 1595, "cache_tensor_bytes": null, "serialized_payload_bytes": 1595, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 17029.7132920241, "decode_payload_ms": 0.0, "receiver_prefill_ms": 189.33529197238386, "ttft_ms": 44.18208298739046, "decode_ms": 195.17187506426126}, "total_wall_ms": 430.00350007787347, "ttft_ms": 44.18208298739046, "input_usage": {"receiver_prompt_tokens": 614, "sender_input_tokens": 1900, "sender_output_tokens": 512, "sender_prefill_ms": 444.6433329721913, "sender_ttft_ms": 42.203875025734305, "sender_decode_ms": 16540.646250010468}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6561218560, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6904610296, "mlx_cache_bytes": 4814004772, "process_rss_bytes": 6561218560}, "peak_prefill_memory": null, "peak_decode_memory": 6904610296, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2670c7b1d77f11e98aea66022431c5d5cd7ede827d457184e5ccd1710c1c584e", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
25 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "411f61984792a78bf25b5893", "attempt_id": "54a3e7485b18", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:57.740971Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 443, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 130.85904193576425, "ttft_ms": 37.889541941694915, "decode_ms": 35.92687495984137}, "total_wall_ms": 206.39716705773026, "ttft_ms": 37.889541941694915, "input_usage": {"receiver_prompt_tokens": 443, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6562988032, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6738705496, "mlx_cache_bytes": 5622825032, "process_rss_bytes": 6562988032}, "peak_prefill_memory": null, "peak_decode_memory": 6738705496, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
26 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3cc9446b3ad102c3baf6af25", "attempt_id": "e9c37c9c8d05", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:57.969766Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 448, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 144.1592500777915, "ttft_ms": 20.462832995690405, "decode_ms": 41.42437502741814}, "total_wall_ms": 207.5935839675367, "ttft_ms": 20.462832995690405, "input_usage": {"receiver_prompt_tokens": 448, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6563053568, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6741605464, "mlx_cache_bytes": 5873107016, "process_rss_bytes": 6563053568}, "peak_prefill_memory": null, "peak_decode_memory": 6741605464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
27 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c27ce71f958b1a8eab837012", "attempt_id": "34724b1a5730", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:58.201511Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 448, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 170.96658307127655, "ttft_ms": 44.05787494033575, "decode_ms": 87.86874997895211}, "total_wall_ms": 304.1407499695197, "ttft_ms": 44.05787494033575, "input_usage": {"receiver_prompt_tokens": 448, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6563119104, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6741605464, "mlx_cache_bytes": 5873107016, "process_rss_bytes": 6563119104}, "peak_prefill_memory": null, "peak_decode_memory": 6741605464, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "58.0820", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b38eb8a0b5e545425ac83b4aa75ce5fda3dec19e0fb0d34757f1276e1e89f55b"}
28 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "04038b21771c154b41a48952", "attempt_id": "c059118314e3", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:43:58.528210Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 445, "message_tokens": 331, "message_utf8_bytes": 922, "cache_tensor_bytes": null, "serialized_payload_bytes": 922, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 19464.825083035976, "decode_payload_ms": 0.0, "receiver_prefill_ms": 147.983834031038, "ttft_ms": 45.002790982834995, "decode_ms": 24.279332952573895}, "total_wall_ms": 218.7393750064075, "ttft_ms": 45.002790982834995, "input_usage": {"receiver_prompt_tokens": 445, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 487.87766601890326, "sender_ttft_ms": 50.601374940015376, "sender_decode_ms": 18924.01612503454}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6563135488, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6762527832, "mlx_cache_bytes": 6031409224, "process_rss_bytes": 6563135488}, "peak_prefill_memory": null, "peak_decode_memory": 6762527832, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e137340f68268b678688cd455e9b3cd6eb172e1f73407cc86ea551d8a3db9112", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
29 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "84a6deed072a53ee196c6aee", "attempt_id": "3ee33f336d99", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:19.121617Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 918, "message_tokens": 821, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1496, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 220.54516593925655, "ttft_ms": 24.7326249955222, "decode_ms": 286.289207986556}, "total_wall_ms": 533.5963340476155, "ttft_ms": 24.7326249955222, "input_usage": {"receiver_prompt_tokens": 918, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6564102144, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6962990168, "mlx_cache_bytes": 6599819336, "process_rss_bytes": 6564102144}, "peak_prefill_memory": null, "peak_decode_memory": 6962990168, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1496, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "20e51e9edfce0f1f7bbb42564f4a801681629d693f39bdfd2f0fa18174b9872c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
30 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "9ae727a39e3113fae8cdea21", "attempt_id": "5576328117db", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:19.675066Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 927, "message_tokens": 825, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1498, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 253.4893749980256, "ttft_ms": 33.21612498257309, "decode_ms": 33.41083403211087}, "total_wall_ms": 321.7156248865649, "ttft_ms": 33.21612498257309, "input_usage": {"receiver_prompt_tokens": 927, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6564380672, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6967446616, "mlx_cache_bytes": 7055458376, "process_rss_bytes": 6564380672}, "peak_prefill_memory": null, "peak_decode_memory": 6967446616, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1498, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "dd58c6723f7a488b688a9800bd53ab4c41b6262ba448a078f439a766616620c8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
31 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "55543c366c72ffdbbd39c8af", "attempt_id": "66686a257fbb", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:20.017815Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 911, "message_tokens": 809, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1498, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 246.01679190527648, "ttft_ms": 32.576250028796494, "decode_ms": 151.59720892552286}, "total_wall_ms": 432.2824589908123, "ttft_ms": 32.576250028796494, "input_usage": {"receiver_prompt_tokens": 911, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6564429824, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6964415564, "mlx_cache_bytes": 7480819784, "process_rss_bytes": 6564429824}, "peak_prefill_memory": null, "peak_decode_memory": 6964415564, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1498, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e1819d55abb157660a30ec7e8741d5ff9892523ece3b510e1752a6800ca62402", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
32 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "99d6c56f377436971f9e4433", "attempt_id": "d7f4cc777047", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:20.470661Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 924, "message_tokens": 825, "message_utf8_bytes": 816, "cache_tensor_bytes": null, "serialized_payload_bytes": 816, "side_information_bytes": 0, "fetched_bytes": 1498, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20351.395833073184, "decode_payload_ms": 0.0, "receiver_prefill_ms": 265.9725840203464, "ttft_ms": 36.55616601463407, "decode_ms": 28.996124980039895}, "total_wall_ms": 333.160417038016, "ttft_ms": 36.55616601463407, "input_usage": {"receiver_prompt_tokens": 924, "sender_input_tokens": 1930, "sender_output_tokens": 512, "sender_prefill_ms": 592.018208000809, "sender_ttft_ms": 40.81787494942546, "sender_decode_ms": 19716.19079192169}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6564495360, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7001082968, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6564495360}, "peak_prefill_memory": null, "peak_decode_memory": 7001082968, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1498, "retrieved_refs": 6}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "dd58c6723f7a488b688a9800bd53ab4c41b6262ba448a078f439a766616620c8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7b6538619254e7b397e82909", "attempt_id": "499765337f31", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:20.830950Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1862, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 498.52987495251, "ttft_ms": 36.994458991102874, "decode_ms": 260.9179160790518}, "total_wall_ms": 800.1195830293, "ttft_ms": 36.994458991102874, "input_usage": {"receiver_prompt_tokens": 1862}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6569115648, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7120900424, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569115648}, "peak_prefill_memory": null, "peak_decode_memory": 7120900424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
34 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "93b4d8e4431a75f0233ad746", "attempt_id": "545588961097", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:21.651547Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 509.5061670290306, "ttft_ms": 49.28991699125618, "decode_ms": 315.8052080543712}, "total_wall_ms": 877.6191669749096, "ttft_ms": 49.28991699125618, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6569164800, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569164800}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
35 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "88d68cf77d7e11beadc7266c", "attempt_id": "2645dbd8e95d", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:22.553539Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1867, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 450.3524579340592, "ttft_ms": 31.306082964874804, "decode_ms": 152.0601250231266}, "total_wall_ms": 636.4715420641005, "ttft_ms": 31.306082964874804, "input_usage": {"receiver_prompt_tokens": 1867}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6569181184, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7122260316, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569181184}, "peak_prefill_memory": null, "peak_decode_memory": 7122260316, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
36 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e2b2bcfa03ca62ad479ecb48", "attempt_id": "f2ba1f71926d", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:23.209456Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 3.5135135135135136}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 1864, "message_tokens": 1766, "message_utf8_bytes": 7280, "cache_tensor_bytes": null, "serialized_payload_bytes": 2072, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 7.439083070494235, "decode_payload_ms": 0.5127090262249112, "receiver_prefill_ms": 491.95970804430544, "ttft_ms": 44.978291960433125, "decode_ms": 222.31799992732704}, "total_wall_ms": 762.6348329940811, "ttft_ms": 44.978291960433125, "input_usage": {"receiver_prompt_tokens": 1864}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6569197568, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7179080028, "mlx_cache_bytes": 7677526088, "process_rss_bytes": 6569197568}, "peak_prefill_memory": null, "peak_decode_memory": 7179080028, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
37 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "cc589ec7a25b40c391448878", "attempt_id": "8d1266ebf492", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:24.515315Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 6.305875023826957, "receiver_prefill_ms": 0.0, "query_prefill_ms": 72.47437501791865, "ttft_ms": 72.47437501791865, "decode_ms": 261.442834045738}, "total_wall_ms": 341.9492499670014, "ttft_ms": 72.47437501791865, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6991708160, "backend_memory": {"mlx_active_bytes": 7085356040, "mlx_peak_bytes": 7148451120, "mlx_cache_bytes": 8003223625, "process_rss_bytes": 6991708160}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7148451120, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
38 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "be00ef5059d78d7afef9989a", "attempt_id": "54c814cd5911", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:24.879707Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 5.109624937176704, "receiver_prefill_ms": 0.0, "query_prefill_ms": 77.48341606929898, "ttft_ms": 77.48341606929898, "decode_ms": 384.95275005698204}, "total_wall_ms": 469.2965829744935, "ttft_ms": 77.48341606929898, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6991921152, "backend_memory": {"mlx_active_bytes": 7086273544, "mlx_peak_bytes": 7152973104, "mlx_cache_bytes": 8294957129, "process_rss_bytes": 6991921152}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152973104, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
39 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "5dbb517dc1e39027d47ce53d", "attempt_id": "67c316ecb86b", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:25.371676Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 4.740834003314376, "receiver_prefill_ms": 0.0, "query_prefill_ms": 61.675542034208775, "ttft_ms": 61.675542034208775, "decode_ms": 163.4873750153929}, "total_wall_ms": 231.41041700728238, "ttft_ms": 61.675542034208775, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6991953920, "backend_memory": {"mlx_active_bytes": 7085356040, "mlx_peak_bytes": 7152973104, "mlx_cache_bytes": 8295874633, "process_rss_bytes": 6991953920}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152973104, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
40 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e30c2d93f6504db52f420ff5", "attempt_id": "201b24eb535f", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:25.623900Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870660104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433294, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 522.5020839134231, "doc_prefill_ms": 437.422874965705, "compress_ms": 10.527916951104999, "serialize_ms": 70.8771669305861, "load_ms": 8.166083018295467, "receiver_prefill_ms": 0.0, "query_prefill_ms": 58.05266601964831, "ttft_ms": 58.05266601964831, "decode_ms": 214.86158296465874}, "total_wall_ms": 282.6327499933541, "ttft_ms": 58.05266601964831, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6992019456, "backend_memory": {"mlx_active_bytes": 7085356040, "mlx_peak_bytes": 7152416048, "mlx_cache_bytes": 8315551817, "process_rss_bytes": 6992019456}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152416048, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "fa7deb82387df79ba9c62b52861e9729c29f9571c7ce3cc4593b01366fba1eb9", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
41 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3bdce55ca1c203e6cb369146", "attempt_id": "cfdd9fae6058", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:25.999220Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 38.35595794953406, "receiver_prefill_ms": 0.0, "query_prefill_ms": 61.90962495747954, "ttft_ms": 61.90962495747954, "decode_ms": 312.5291250180453}, "total_wall_ms": 414.4973329966888, "ttft_ms": 61.90962495747954, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7202373632, "backend_memory": {"mlx_active_bytes": 7085192200, "mlx_peak_bytes": 7148139824, "mlx_cache_bytes": 8315715657, "process_rss_bytes": 7202373632}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7148139824, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
42 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "da49bbe0ee1937a1d6cea01b", "attempt_id": "262b74cca3b0", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:26.434479Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 27.08279201760888, "receiver_prefill_ms": 0.0, "query_prefill_ms": 36.74579202197492, "ttft_ms": 36.74579202197492, "decode_ms": 428.5999160492793}, "total_wall_ms": 493.9215829363093, "ttft_ms": 36.74579202197492, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7202406400, "backend_memory": {"mlx_active_bytes": 7086109704, "mlx_peak_bytes": 7152809264, "mlx_cache_bytes": 8314798153, "process_rss_bytes": 7202406400}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152809264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
43 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "16b714f92a451743570b3b31", "attempt_id": "cf8fc0d51247", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:26.948504Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 26.593707967549562, "receiver_prefill_ms": 0.0, "query_prefill_ms": 71.3852500775829, "ttft_ms": 71.3852500775829, "decode_ms": 116.4509579539299}, "total_wall_ms": 215.88545804843307, "ttft_ms": 71.3852500775829, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7202455552, "backend_memory": {"mlx_active_bytes": 7085192200, "mlx_peak_bytes": 7152809264, "mlx_cache_bytes": 8315715657, "process_rss_bytes": 7202455552}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152809264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
44 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3ab2375116fc39604b6bb3dd", "attempt_id": "0e62b672bf28", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:27.184808Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6870496264}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": "63d490dc6cecd04f", "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 209420288, "serialized_payload_bytes": 209433322, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 72.577791986987, "doc_prefill_ms": 437.422874965705, "compress_ms": 5.062707932665944, "serialize_ms": 67.35641695559025, "load_ms": 29.1128340177238, "receiver_prefill_ms": 0.0, "query_prefill_ms": 56.22645898256451, "ttft_ms": 56.22645898256451, "decode_ms": 231.56520805787295}, "total_wall_ms": 318.4051669668406, "ttft_ms": 56.22645898256451, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7202471936, "backend_memory": {"mlx_active_bytes": 7085192200, "mlx_peak_bytes": 7152219440, "mlx_cache_bytes": 8315715657, "process_rss_bytes": 7202471936}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 7152219440, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c36a83ab14f375c59623a755718dc5b0bdfcef7c346cb20d001ff25b80cf6a92", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
45 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "99a3adca6917879f925d6786", "attempt_id": "a920ac2a6176", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:27.586430Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 13.677041977643967, "receiver_prefill_ms": 0.0, "query_prefill_ms": 217.12250006385148, "ttft_ms": 217.12250006385148, "decode_ms": 343.93254097085446}, "total_wall_ms": 576.2448329478502, "ttft_ms": 217.12250006385148, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7316357120, "backend_memory": {"mlx_active_bytes": 6888567816, "mlx_peak_bytes": 6964742096, "mlx_cache_bytes": 8847362761, "process_rss_bytes": 7316357120}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6964742096, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
46 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8145f501bf31191588311a59", "attempt_id": "d17cb71f1d20", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:28.184216Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 3.7757500540465117, "receiver_prefill_ms": 0.0, "query_prefill_ms": 60.53770799189806, "ttft_ms": 60.53770799189806, "decode_ms": 339.0390001004562}, "total_wall_ms": 405.1002920605242, "ttft_ms": 60.53770799189806, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7316570112, "backend_memory": {"mlx_active_bytes": 6889485320, "mlx_peak_bytes": 6971168592, "mlx_cache_bytes": 9017721641, "process_rss_bytes": 7316570112}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6971168592, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
47 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "9e09d4a66b95560c536e4bb6", "attempt_id": "3398867c7cff", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:28.612313Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 7.927541038952768, "receiver_prefill_ms": 0.0, "query_prefill_ms": 89.23725003842264, "ttft_ms": 89.23725003842264, "decode_ms": 179.2399160331115}, "total_wall_ms": 278.06874993257225, "ttft_ms": 89.23725003842264, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7316586496, "backend_memory": {"mlx_active_bytes": 6888502280, "mlx_peak_bytes": 6971171536, "mlx_cache_bytes": 9018704681, "process_rss_bytes": 7316586496}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6971171536, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
48 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c936609e91653935523c0908", "attempt_id": "43b1cfa6952a", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:28.911150Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6773814280}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 111254528, "serialized_payload_bytes": 111281646, "side_information_bytes": 27118, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 61.96125003043562, "doc_prefill_ms": 437.422874965705, "compress_ms": 30.956957954913378, "serialize_ms": 30.65370803233236, "load_ms": 4.78758302051574, "receiver_prefill_ms": 0.0, "query_prefill_ms": 39.28579203784466, "ttft_ms": 39.28579203784466, "decode_ms": 184.25929197110236}, "total_wall_ms": 230.09091592393816, "ttft_ms": 39.28579203784466, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7316586496, "backend_memory": {"mlx_active_bytes": 6888567816, "mlx_peak_bytes": 6971415216, "mlx_cache_bytes": 9045216425, "process_rss_bytes": 7316586496}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6971415216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c81fb07470693068b7485351f664d38e900f7f96b9ec080e540dd33f44152a99", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
49 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "7d6acf26fc79182f5dbb1ed8", "attempt_id": "15136fb321a2", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:29.244120Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 3.18250001873821, "receiver_prefill_ms": 0.0, "query_prefill_ms": 185.57591596618295, "ttft_ms": 185.57591596618295, "decode_ms": 295.966292032972}, "total_wall_ms": 486.3574579358101, "ttft_ms": 185.57591596618295, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7371276288, "backend_memory": {"mlx_active_bytes": 6782612488, "mlx_peak_bytes": 6847246256, "mlx_cache_bytes": 9327865001, "process_rss_bytes": 7371276288}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6847246256, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
50 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c564af55b1354e855d13ebb5", "attempt_id": "d0c64cba8a9d", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:29.752314Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 2.182958065532148, "receiver_prefill_ms": 0.0, "query_prefill_ms": 63.19291703402996, "ttft_ms": 63.19291703402996, "decode_ms": 353.3098748885095}, "total_wall_ms": 420.24579201824963, "ttft_ms": 63.19291703402996, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7371325440, "backend_memory": {"mlx_active_bytes": 6782645256, "mlx_peak_bytes": 6854042320, "mlx_cache_bytes": 9327832233, "process_rss_bytes": 7371325440}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6854042320, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
51 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "5f2dc6cc5e94630f30047acf", "attempt_id": "ef05ade2e72f", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:30.194955Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 1.280625001527369, "receiver_prefill_ms": 0.0, "query_prefill_ms": 42.38108394201845, "ttft_ms": 42.38108394201845, "decode_ms": 163.35366701241583}, "total_wall_ms": 208.8556250091642, "ttft_ms": 42.38108394201845, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7371358208, "backend_memory": {"mlx_active_bytes": 6782579720, "mlx_peak_bytes": 6854042320, "mlx_cache_bytes": 9327897769, "process_rss_bytes": 7371358208}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6854042320, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
52 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "72933f8a71427c88fc872a97", "attempt_id": "4f258f174b74", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:30.427208Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 58899456, "serialized_payload_bytes": 58926513, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 79.94737499393523, "doc_prefill_ms": 437.422874965705, "compress_ms": 51.74345790874213, "serialize_ms": 27.822209056466818, "load_ms": 2.032542019151151, "receiver_prefill_ms": 0.0, "query_prefill_ms": 54.19116606935859, "ttft_ms": 54.19116606935859, "decode_ms": 194.77229204494506}, "total_wall_ms": 252.6729580713436, "ttft_ms": 54.19116606935859, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7371374592, "backend_memory": {"mlx_active_bytes": 6782579720, "mlx_peak_bytes": 6853991280, "mlx_cache_bytes": 9328159913, "process_rss_bytes": 7371374592}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6853991280, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4180fb98d26b8f9bc2c30120aea76b31662b7d5e9c4b045f5c9627ed7b0df3a6", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
53 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "4a990a3bbc163af0254f4da9", "attempt_id": "decdff1d7a8a", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:30.835010Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 0.7832079427316785, "receiver_prefill_ms": 0.0, "query_prefill_ms": 48.5515829641372, "ttft_ms": 48.5515829641372, "decode_ms": 492.50808299984783}, "total_wall_ms": 543.5889170039445, "ttft_ms": 48.5515829641372, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 12}, "provider_cached_usage": null, "process_memory": 7392919552, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 9428331689, "process_rss_bytes": 7392919552}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "There is no Torsel-5 in the text.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "9692259f44dd18dd1cdbacd4f40fc5c86163138fa40a692cba37d5829d9efdae"}
54 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "edb167c9d4a110a48c5571db", "attempt_id": "9bd754b97ad0", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:31.403191Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 4.029666073620319, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.984708088450134, "ttft_ms": 59.984708088450134, "decode_ms": 240.85499998182058}, "total_wall_ms": 306.52774998452514, "ttft_ms": 59.984708088450134, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7392854016, "backend_memory": {"mlx_active_bytes": 6772323336, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9444846761, "process_rss_bytes": 7392854016}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
55 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "fd54709862c7fe9cda0b03ce", "attempt_id": "ae351568c289", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:31.732096Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 1.343625015579164, "receiver_prefill_ms": 0.0, "query_prefill_ms": 50.66633294336498, "ttft_ms": 50.66633294336498, "decode_ms": 127.93991691432893}, "total_wall_ms": 181.72441702336073, "ttft_ms": 50.66633294336498, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7392903168, "backend_memory": {"mlx_active_bytes": 6772192264, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9444977833, "process_rss_bytes": 7392903168}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not available.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "469147b9096ca76595ba03d6e8e849959829de7be9f36bbb9e437b35a34af4c5"}
56 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "f0e55404bf4c08124bf59b7e", "attempt_id": "5e01a4805fd3", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:31.936141Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 56.43370805773884}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310610, "side_information_bytes": 12882, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 132.07170902751386, "doc_prefill_ms": 437.422874965705, "compress_ms": 56.50170799344778, "serialize_ms": 75.29904192779213, "load_ms": 1.520541962236166, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.29674994852394, "ttft_ms": 31.29674994852394, "decode_ms": 252.26949993520975}, "total_wall_ms": 286.5531659917906, "ttft_ms": 31.29674994852394, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 7392477184, "backend_memory": {"mlx_active_bytes": 6772192264, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 9444977833, "process_rss_bytes": 7392477184}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "AC-59L-3161", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "4bede118afa323ca6bb797abb36c340ea9d9edcd57e651802639f006b0fd8a3f", "output_hash": "ad64332dcd39755f6897a634ba23cec1a004826ea0ea9dabd0093a9c2ec190b0"}
57 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "52c48b9797ca5c06552b1531", "attempt_id": "6091f1f0713c", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:32.302798Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 1.3935830211266875, "receiver_prefill_ms": 0.0, "query_prefill_ms": 57.18616594094783, "ttft_ms": 57.18616594094783, "decode_ms": 1790.9953750204295}, "total_wall_ms": 1851.3322500512004, "ttft_ms": 57.18616594094783, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7445053440, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 7445053440}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n\n\n\n\n\n\n\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n}\n\n}\n\n\n\n//}\n\n//////////////////////", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "ccccfe0de485e198bddd0417f4df11b4b4717882fdaf107fd56e690b6fa5f10b"}
58 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "24076303fc2fee00bc3a854e", "attempt_id": "a133016fcd07", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:34.175581Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 3.364000003784895, "receiver_prefill_ms": 0.0, "query_prefill_ms": 44.80479098856449, "ttft_ms": 44.80479098856449, "decode_ms": 1871.8788340920582}, "total_wall_ms": 1922.2292910562828, "ttft_ms": 44.80479098856449, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7445102592, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 7445102592}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n\n\n}\n\n}\n\n}\n\n}\n\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "8a98bf1cdbfff3febaa313db2ee830212976ff44521e87d9f91743b75c411c7c"}
59 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "f481d30b9909473a1d70d9a6", "attempt_id": "dc9da1b7970d", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:36.118241Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 2.0380420610308647, "receiver_prefill_ms": 0.0, "query_prefill_ms": 19.68616701196879, "ttft_ms": 19.68616701196879, "decode_ms": 1811.1777500016615}, "total_wall_ms": 1834.7261250019073, "ttft_ms": 19.68616701196879, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 7445102592, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 7445102592}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n//////////////////////////////////////////////////////////////////////////////////////////", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "9c1c85d0af791dd782b2fc598b4c2453d6625d60ad58bacf00b9ab5c4e254186"}
60 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c0bd23fcdf788f4e76591000", "attempt_id": "6bc2cb9bbadf", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:37.975107Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1826, "keep": 456, "compress_ms": 9.820375009439886}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310605, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 60.8185000019148, "doc_prefill_ms": 437.422874965705, "compress_ms": 9.860167047008872, "serialize_ms": 50.75112509075552, "load_ms": 5.221916944719851, "receiver_prefill_ms": 0.0, "query_prefill_ms": 48.19733300246298, "ttft_ms": 48.19733300246298, "decode_ms": 1727.6302080135792}, "total_wall_ms": 1782.6562089612707, "ttft_ms": 48.19733300246298, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 48}, "provider_cached_usage": null, "process_memory": 6908248064, "backend_memory": {"mlx_active_bytes": 6775993352, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 9754963113, "process_rss_bytes": 6908248064}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "}\n\n}\n\n}\n\n//////////////////////////////////////////////////////////////////////////////////////////", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "00809d6606d7fc50c17f04bd8a4a3707958f32db62bac946b1c32fc3e676b47f", "output_hash": "9c1c85d0af791dd782b2fc598b4c2453d6625d60ad58bacf00b9ab5c4e254186"}
61 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c9bb5f57c39cd9b828c7092a", "attempt_id": "e9dc8ce0e20e", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:39.778208Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 36, "compress_ms": 223.24320790357888}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 245.0225839857012, "doc_prefill_ms": 437.422874965705, "compress_ms": 224.6504999930039, "serialize_ms": 20.149499992839992, "load_ms": 2.4289999855682254, "receiver_prefill_ms": 0.0, "query_prefill_ms": 38.21954189334065, "ttft_ms": 38.21954189334065, "decode_ms": 227.5533340871334}, "total_wall_ms": 515.092583023943, "ttft_ms": 38.21954189334065, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6921027584, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 10017287337, "process_rss_bytes": 6921027584}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
62 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ed4dca7d54c8d06691e35017", "attempt_id": "7f0d9603a35d", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:40.314694Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 162.85679198335856}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 192.2094589099288, "doc_prefill_ms": 437.422874965705, "compress_ms": 164.35245797038078, "serialize_ms": 27.638208004646003, "load_ms": 2.581541077233851, "receiver_prefill_ms": 0.0, "query_prefill_ms": 46.956875012256205, "ttft_ms": 46.956875012256205, "decode_ms": 314.66949998866767}, "total_wall_ms": 558.5637079784647, "ttft_ms": 46.956875012256205, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6911475712, "backend_memory": {"mlx_active_bytes": 6772323336, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10036014249, "process_rss_bytes": 6911475712}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
63 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "55d7c75cb9ebbc123fa690f9", "attempt_id": "957cb1da168c", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:40.897428Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 147.52570795826614}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 162.3099580174312, "doc_prefill_ms": 437.422874965705, "compress_ms": 149.2385419551283, "serialize_ms": 12.80104205943644, "load_ms": 1.1120840208604932, "receiver_prefill_ms": 0.0, "query_prefill_ms": 41.776582947932184, "ttft_ms": 41.776582947932184, "decode_ms": 141.10079209785908}, "total_wall_ms": 348.03975000977516, "ttft_ms": 41.776582947932184, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6913359872, "backend_memory": {"mlx_active_bytes": 6772192264, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10036145321, "process_rss_bytes": 6913359872}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
64 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e477b9a1d55a7ccb17c67451", "attempt_id": "4ae83690bd30", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:41.267533Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 38, "compress_ms": 129.34766698163003}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 146.5985419927165, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.98404195625335, "serialize_ms": 15.367292100563645, "load_ms": 2.997333067469299, "receiver_prefill_ms": 0.0, "query_prefill_ms": 51.74429202452302, "ttft_ms": 51.74429202452302, "decode_ms": 221.70766699127853}, "total_wall_ms": 425.10620795655996, "ttft_ms": 51.74429202452302, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6915211264, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 10043419817, "process_rss_bytes": 6915211264}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
65 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b46d4fcd3e2cacc4f4171eda", "attempt_id": "4d532bb7e02d", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:41.858815Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 0.9596669115126133, "receiver_prefill_ms": 0.0, "query_prefill_ms": 27.27637509815395, "ttft_ms": 27.27637509815395, "decode_ms": 24.090874940156937}, "total_wall_ms": 54.03216590639204, "ttft_ms": 27.27637509815395, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6770488328, "mlx_peak_bytes": 6811006264, "mlx_cache_bytes": 10065439989, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6811006264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
66 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d97a8ce2c81cfb32ae8dc73a", "attempt_id": "745244fed267", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:41.934154Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 1.297999988310039, "receiver_prefill_ms": 0.0, "query_prefill_ms": 52.7221669908613, "ttft_ms": 52.7221669908613, "decode_ms": 41.93408403079957}, "total_wall_ms": 97.69941691774875, "ttft_ms": 52.7221669908613, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10063867125, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
67 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "1899008e558dfaa91056372a", "attempt_id": "5a1814d043d3", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:42.050982Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 1.8834170186892152, "receiver_prefill_ms": 0.0, "query_prefill_ms": 44.49391597881913, "ttft_ms": 44.49391597881913, "decode_ms": 37.344375043176115}, "total_wall_ms": 85.45716595835984, "ttft_ms": 44.49391597881913, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6772061192, "mlx_peak_bytes": 6817002808, "mlx_cache_bytes": 10063867125, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6817002808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
68 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8bf08c1d45fa158cabe9c701", "attempt_id": "e50e364f8914", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:42.158671Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 19, "compress_ms": 130.59799990151078}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310651, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 142.4080830765888, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.77891699504107, "serialize_ms": 11.471665930002928, "load_ms": 2.5057089515030384, "receiver_prefill_ms": 0.0, "query_prefill_ms": 35.62441701069474, "ttft_ms": 35.62441701069474, "decode_ms": 42.37329191528261}, "total_wall_ms": 82.29766692966223, "ttft_ms": 35.62441701069474, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6917292032, "backend_memory": {"mlx_active_bytes": 6770488328, "mlx_peak_bytes": 6815839544, "mlx_cache_bytes": 10065439989, "process_rss_bytes": 6917292032}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6815839544, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "5fb14777456d937b3f49bcee3ce7cc4906bf362fefc39cb2f8070a5576ea10ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
69 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "6644725e8322781a5ad9d3f9", "attempt_id": "1dbb2113f66c", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:42.263727Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 36, "compress_ms": 155.98812501411885}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310621, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 179.18008391279727, "doc_prefill_ms": 437.422874965705, "compress_ms": 157.55825000815094, "serialize_ms": 21.383208106271923, "load_ms": 3.9268749533221126, "receiver_prefill_ms": 0.0, "query_prefill_ms": 872.3779579158872, "ttft_ms": 872.3779579158872, "decode_ms": 265.98583289887756}, "total_wall_ms": 1323.2823330909014, "ttft_ms": 872.3779579158872, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6851657728, "backend_memory": {"mlx_active_bytes": 6771045384, "mlx_peak_bytes": 6839888636, "mlx_cache_bytes": 10584059665, "process_rss_bytes": 6851657728}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6839888636, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2b623b234e50bb45b3d64ddb4e5fc624d2e8d987140d846139a2f1fc5933cbce", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
70 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "bbc8aae14960288c98be5b42", "attempt_id": "1dc7fe1a4905", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:43.609212Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 154.456916032359}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310619, "side_information_bytes": 12891, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 177.84450005274266, "doc_prefill_ms": 437.422874965705, "compress_ms": 156.14125004503876, "serialize_ms": 21.469792001880705, "load_ms": 1.3804170303046703, "receiver_prefill_ms": 0.0, "query_prefill_ms": 938.1341250846162, "ttft_ms": 938.1341250846162, "decode_ms": 359.9463750142604}, "total_wall_ms": 1478.9378329878673, "ttft_ms": 938.1341250846162, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 6911180800, "backend_memory": {"mlx_active_bytes": 6771897352, "mlx_peak_bytes": 6844136300, "mlx_cache_bytes": 10614410077, "process_rss_bytes": 6911180800}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6844136300, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "848149b18f72edc487ee684ea55641b98ac2d675e22c288ffa0f3a81f652f67e", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
71 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "5f17d37ee4da2fa8a0c22e08", "attempt_id": "1b3d099d25e4", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:45.112131Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 41, "compress_ms": 129.18295804411173}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310621, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 145.54508298169822, "doc_prefill_ms": 437.422874965705, "compress_ms": 130.7377500925213, "serialize_ms": 14.53899999614805, "load_ms": 1.1376250768080354, "receiver_prefill_ms": 0.0, "query_prefill_ms": 1037.930167047307, "ttft_ms": 1037.930167047307, "decode_ms": 149.28004110697657}, "total_wall_ms": 1335.5639590881765, "ttft_ms": 1037.930167047307, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6947028992, "backend_memory": {"mlx_active_bytes": 6771274760, "mlx_peak_bytes": 6844136300, "mlx_cache_bytes": 10615032669, "process_rss_bytes": 6947028992}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6844136300, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2b623b234e50bb45b3d64ddb4e5fc624d2e8d987140d846139a2f1fc5933cbce", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
72 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "fc3835ebfc431e41ae846d63", "attempt_id": "9f4834358d02", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:46.471228Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1826, "keep": 456, "window_tokens": 38, "compress_ms": 139.58499999716878}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 52297728, "serialized_payload_bytes": 52310621, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 156.92758397199214, "doc_prefill_ms": 437.422874965705, "compress_ms": 141.24699996318668, "serialize_ms": 15.446165925823152, "load_ms": 1.5408750623464584, "receiver_prefill_ms": 0.0, "query_prefill_ms": 883.123290957883, "ttft_ms": 883.123290957883, "decode_ms": 175.33191700931638}, "total_wall_ms": 1218.847917043604, "ttft_ms": 883.123290957883, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6989185024, "backend_memory": {"mlx_active_bytes": 6771012616, "mlx_peak_bytes": 6841477948, "mlx_cache_bytes": 10621881181, "process_rss_bytes": 6989185024}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6841477948, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2b623b234e50bb45b3d64ddb4e5fc624d2e8d987140d846139a2f1fc5933cbce", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
73 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8effa93c0f06a54d68593ab3", "attempt_id": "b758fe81f510", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:47.848242Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 36, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 3.6407920997589827, "receiver_prefill_ms": 0.0, "query_prefill_ms": 36.66383307427168, "ttft_ms": 36.66383307427168, "decode_ms": 99.13308289833367}, "total_wall_ms": 141.15158305503428, "ttft_ms": 36.66383307427168, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 36}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7049773056, "backend_memory": {"mlx_active_bytes": 6776222728, "mlx_peak_bytes": 6840512432, "mlx_cache_bytes": 10683583325, "process_rss_bytes": 7049773056}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6840512432, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Not found.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "0802559db1375af3ff5caabba71acea1d6299f1a7fc64b6a5024f19cbd33b72f"}
74 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d4eb8890a5e642f7ab6000aa", "attempt_id": "d61c0956fdb3", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:48.013724Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 3.137958934530616, "receiver_prefill_ms": 0.0, "query_prefill_ms": 52.22874996252358, "ttft_ms": 52.22874996252358, "decode_ms": 385.93683403450996}, "total_wall_ms": 442.7836670074612, "ttft_ms": 52.22874996252358, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7049789440, "backend_memory": {"mlx_active_bytes": 6778057736, "mlx_peak_bytes": 6836265264, "mlx_cache_bytes": 10681748317, "process_rss_bytes": 7049789440}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6836265264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lot 61-C, Schefferville.", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.8571428571428571}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "5280344bb130ba2ce68331d4c5deadfa47c813ecd5c007f5616e1ff44aa0daea"}
75 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2d065ff453763a5f4c3731f7", "attempt_id": "999a8ae40b7c", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:48.479776Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 41, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 6.236125016584992, "receiver_prefill_ms": 0.0, "query_prefill_ms": 53.27745899558067, "ttft_ms": 53.27745899558067, "decode_ms": 147.6881660055369}, "total_wall_ms": 208.8929999154061, "ttft_ms": 53.27745899558067, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 41}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 7049805824, "backend_memory": {"mlx_active_bytes": 6776222728, "mlx_peak_bytes": 6836265264, "mlx_cache_bytes": 10683583325, "process_rss_bytes": 7049805824}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6836265264, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "78.0424", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "a6b0ea8de12cdf1084cdbceb42fd84946cd25a872f44d1f9a0846f3cf0f98534"}
76 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0ed88b91f50021037e7b63fa", "attempt_id": "10349d5e5002", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:48.711103Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1826, "keep": 913, "compress_ms": 49.07041706610471, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 4, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1766, "source_tokens_receiver": 1766, "query_tokens": 38, "message_tokens": 1826, "message_utf8_bytes": 7280, "cache_tensor_bytes": 55627264, "serialized_payload_bytes": 55654040, "side_information_bytes": 26776, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 135.99824998527765, "doc_prefill_ms": 437.422874965705, "compress_ms": 104.20516703743488, "serialize_ms": 31.361375004053116, "load_ms": 2.9613750521093607, "receiver_prefill_ms": 0.0, "query_prefill_ms": 31.615874962881207, "ttft_ms": 31.615874962881207, "decode_ms": 254.29916696157306}, "total_wall_ms": 290.79958400689065, "ttft_ms": 31.615874962881207, "input_usage": {"receiver_prefix_tokens": 1826, "dense_cache_bytes": 209420288, "query_tokens": 38}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 7049822208, "backend_memory": {"mlx_active_bytes": 6776222728, "mlx_peak_bytes": 6837017456, "mlx_cache_bytes": 10683583325, "process_rss_bytes": 7049822208}, "peak_prefill_memory": 7112184136, "peak_decode_memory": 6837017456, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Wilhelmina Rousseau", "evidence_artifact": null, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f8fd9dcee318c26cba28b4b502c6b7cf62df7ec624ef536a87cffed1f586c739", "output_hash": "48d8b48be3dac15e3a879049c76d1150b48d5cfd634a79a298cef143ea8a97c3"}
77 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d9b75cee222d17ea15050553", "attempt_id": "670fbabb21dc", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:49.621118Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 95, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.00925001222640276, "decode_payload_ms": 0.0, "receiver_prefill_ms": 63.3784169331193, "ttft_ms": 51.294708042405546, "decode_ms": 1745.2523339306936}, "total_wall_ms": 1860.6292499462143, "ttft_ms": 51.294708042405546, "input_usage": {"receiver_prompt_tokens": 95}, "output_usage": {"output_tokens": 47}, "provider_cached_usage": null, "process_memory": 6802898944, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6519191004, "mlx_cache_bytes": 388432592, "process_rss_bytes": 6802898944}, "peak_prefill_memory": null, "peak_decode_memory": 6519191004, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n if rows < 0:\n return \"Invalid number of rows\"\n elif rows == 0:\n return 0\n else:\n return rows * 100\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "TypeError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "06087c1270f1bbd4d0fcf8573af49de69f8e400a5022dd76dbda1cb50c96ba93"}
78 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d78132a69217e36ae518ef36", "attempt_id": "60b3015340ec", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:51.527146Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 1898, "message_tokens": 1793, "message_utf8_bytes": 7944, "cache_tensor_bytes": null, "serialized_payload_bytes": 7944, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.01525005791336298, "decode_payload_ms": 0.0, "receiver_prefill_ms": 470.90358298737556, "ttft_ms": 45.215333928354084, "decode_ms": 4239.911125041544}, "total_wall_ms": 4759.3687500339, "ttft_ms": 45.215333928354084, "input_usage": {"receiver_prompt_tokens": 1898}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6803439616, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7130223064, "mlx_cache_bytes": 869458036, "process_rss_bytes": 6803439616}, "peak_prefill_memory": null, "peak_decode_memory": 7130223064, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
79 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e08997cc2feb77c2c937b94a", "attempt_id": "edd940dbeb54", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:44:56.333998Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 1793}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 625, "message_tokens": 515, "message_utf8_bytes": 2185, "cache_tensor_bytes": null, "serialized_payload_bytes": 2185, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 2.051000017672777, "decode_payload_ms": 0.0, "receiver_prefill_ms": 181.25262495595962, "ttft_ms": 51.436083973385394, "decode_ms": 3500.617416924797}, "total_wall_ms": 3734.8810830153525, "ttft_ms": 51.436083973385394, "input_usage": {"receiver_prompt_tokens": 625}, "output_usage": {"output_tokens": 95}, "provider_cached_usage": null, "process_memory": 6804488192, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6872104472, "mlx_cache_bytes": 1329424948, "process_rss_bytes": 6804488192}, "peak_prefill_memory": null, "peak_decode_memory": 6872104472, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n rows = [row for row in rows if row['region'] != 'Sud' and row['returned'] == False and row['units'] >= 2]\n revenue = sum(row['units'] * row['unit_price'] * (1 - row['discount_pct'] / 100) for row in rows)\n revenue *= 1.09975\n return round(revenue, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "2413b1319a63f1b6cd0f54d110715d18bcdb9bb119470d5b741d8c0d7ebff49c", "output_hash": "d6ad48d19daa4028fb032076e444c54864be0c65981308d25e8463d4e1012ab9"}
80 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ddf8979de181cebec5f1827f", "attempt_id": "1c2fdfbd9e2b", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:00.118142Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 501, "n_sentences": 6}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 604, "message_tokens": 498, "message_utf8_bytes": 1553, "cache_tensor_bytes": null, "serialized_payload_bytes": 1553, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.8467919435352087, "decode_payload_ms": 0.0, "receiver_prefill_ms": 181.44291604403406, "ttft_ms": 17.23441598005593, "decode_ms": 3289.129166980274}, "total_wall_ms": 3489.383792039007, "ttft_ms": 17.23441598005593, "input_usage": {"receiver_prompt_tokens": 604}, "output_usage": {"output_tokens": 93}, "provider_cached_usage": null, "process_memory": 6804537344, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6860832280, "mlx_cache_bytes": 1689692724, "process_rss_bytes": 6804537344}, "peak_prefill_memory": null, "peak_decode_memory": 6860832280, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0\n for r in rows:\n if not r['returned'] and r['region'] != 'Sud' and r['units'] >= 2:\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total * (1 + 0.09975), 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "48e30493b93ebcda2339038b160af9a3c10308d66882e2c52b74d425d38bc3ec", "output_hash": "0c7889ee8e7cfd64c00351584d70e550d93bdc704007cedf1a26ae62d0807c38"}
81 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d7584634c71bf21e0d6eecd1", "attempt_id": "70b30cfac8e5", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:03.652108Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 512, "n_sentences": 25}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 618, "message_tokens": 512, "message_utf8_bytes": 2405, "cache_tensor_bytes": null, "serialized_payload_bytes": 2405, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.066208933480084, "decode_payload_ms": 0.0, "receiver_prefill_ms": 154.62425001896918, "ttft_ms": 39.64283294044435, "decode_ms": 3531.6903749480844}, "total_wall_ms": 3728.690249961801, "ttft_ms": 39.64283294044435, "input_usage": {"receiver_prompt_tokens": 618}, "output_usage": {"output_tokens": 98}, "provider_cached_usage": null, "process_memory": 6804602880, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6868844056, "mlx_cache_bytes": 2035198516, "process_rss_bytes": 6804602880}, "peak_prefill_memory": null, "peak_decode_memory": 6868844056, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total_revenue = 0\n for r in rows:\n if r['region'] != 'Sud' and r['units'] >= 2:\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100)\n total_revenue += revenue\n tax = total_revenue * 0.09975\n return round(total_revenue + tax, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1b918631e0fb86c8eeb18ec2fd014d346b49af210fb4d83a68e05ad9997e0880", "output_hash": "386afcb0ebee66414bfab34fc6c287fa4764e3c3d58a941f153b33b00abcdd81"}
82 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "1da1f33c7f89ce4370a99fc6", "attempt_id": "be4c357385b8", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:27.648986Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 622, "message_tokens": 512, "message_utf8_bytes": 1692, "cache_tensor_bytes": null, "serialized_payload_bytes": 1692, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20225.024625076912, "decode_payload_ms": 0.0, "receiver_prefill_ms": 150.09825001470745, "ttft_ms": 49.69558399170637, "decode_ms": 3554.733332945034}, "total_wall_ms": 3756.172207999043, "ttft_ms": 49.69558399170637, "input_usage": {"receiver_prompt_tokens": 622, "sender_input_tokens": 1927, "sender_output_tokens": 512, "sender_prefill_ms": 470.2747920528054, "sender_ttft_ms": 45.99579190835357, "sender_decode_ms": 19706.098709022626}, "output_usage": {"output_tokens": 96}, "provider_cached_usage": null, "process_memory": 6805078016, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6902070808, "mlx_cache_bytes": 3238791244, "process_rss_bytes": 6805078016}, "peak_prefill_memory": null, "peak_decode_memory": 6902070808, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total_revenue = sum(\n (row['units'] * row['unit_price'] * (1 - row['discount_pct'] / 100)) \n for row in rows if not row['returned'] and row['region'] != 'Sud'\n )\n total_revenue *= 1.09975 # Apply 9.975% tax\n return round(total_revenue, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.8, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "ce216383da3fca3914a369e3532324ff4046ab2dfdd40ea6c9eb2f590604f161", "output_hash": "2f4e3ebeefa8303d0e566703fe6dc53c934833e5434ecb87e0da10191df67a70"}
83 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "6ac733142fc67d77e042f09a", "attempt_id": "c339466219ca", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:45:51.782152Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 5, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 526, "message_tokens": 405, "message_utf8_bytes": 1333, "cache_tensor_bytes": null, "serialized_payload_bytes": 1333, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20329.91862494964, "decode_payload_ms": 0.0, "receiver_prefill_ms": 156.54124994762242, "ttft_ms": 29.017125023528934, "decode_ms": 2839.151250082068}, "total_wall_ms": 3026.1665829457343, "ttft_ms": 29.017125023528934, "input_usage": {"receiver_prompt_tokens": 526, "sender_input_tokens": 1957, "sender_output_tokens": 512, "sender_prefill_ms": 517.0142910210416, "sender_ttft_ms": 44.45845796726644, "sender_decode_ms": 19765.974750043824}, "output_usage": {"output_tokens": 78}, "provider_cached_usage": null, "process_memory": 6805225472, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6815022616, "mlx_cache_bytes": 4044957404, "process_rss_bytes": 6805225472}, "peak_prefill_memory": null, "peak_decode_memory": 6815022616, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)", "evidence_artifact": {"n_facts": 5, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "f05cb6c2f9bf2f73fb1dcd0e0639f005f621b57c0475151370086fbe11e43e46", "output_hash": "20782f8c8645c8bef359993e2e9d5715b1b1a2f52039b25419491a0f11425629"}
84 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "87cd1fcac8bca76f19512355", "attempt_id": "f11e12cb7f24", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:14.870978Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 5}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 646, "message_tokens": 540, "message_utf8_bytes": 432, "cache_tensor_bytes": null, "serialized_payload_bytes": 432, "side_information_bytes": 0, "fetched_bytes": 1265, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 20015.72812499944, "decode_payload_ms": 0.0, "receiver_prefill_ms": 220.63470794819295, "ttft_ms": 38.1325000198558, "decode_ms": 2624.743292108178}, "total_wall_ms": 2885.4318329831585, "ttft_ms": 38.1325000198558, "input_usage": {"receiver_prompt_tokens": 646, "sender_input_tokens": 1957, "sender_output_tokens": 512, "sender_prefill_ms": 533.8998339138925, "sender_ttft_ms": 45.12329096905887, "sender_decode_ms": 19434.01129206177}, "output_usage": {"output_tokens": 73}, "provider_cached_usage": null, "process_memory": 6805356544, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6884572780, "mlx_cache_bytes": 4429656304, "process_rss_bytes": 6805356544}, "peak_prefill_memory": null, "peak_decode_memory": 6884572780, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned'] or r['region'] == 'Sud' or r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'])\n return total", "evidence_artifact": {"lookups": 5, "fetched_bytes": 1265, "retrieved_refs": 5}, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "60edc4dbc168ed8b9a202ed54bacfb9f509e83f39b892b5d729d21f65d514068", "output_hash": "11e6c0f73669642380a1848864610e25be94ddbc954c1891d8be076ad8de2943"}
85 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "612cacc00b9ad234007dcb5f", "attempt_id": "477199f5cbef", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:17.807723Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 6.139103554868624}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 1898, "message_tokens": 1793, "message_utf8_bytes": 7944, "cache_tensor_bytes": null, "serialized_payload_bytes": 1294, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.10441604536026716, "decode_payload_ms": 0.018208986148238182, "receiver_prefill_ms": 511.64245803374797, "ttft_ms": 22.179667023010552, "decode_ms": 4115.618415991776}, "total_wall_ms": 4653.617207892239, "ttft_ms": 22.179667023010552, "input_usage": {"receiver_prompt_tokens": 1898}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6805389312, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7130223064, "mlx_cache_bytes": 4429656304, "process_rss_bytes": 6805389312}, "peak_prefill_memory": null, "peak_decode_memory": 7130223064, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
86 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0d74956e2bab90a3256d5dd2", "attempt_id": "463267041c39", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:23.058543Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6873412616}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 212516864, "serialized_payload_bytes": 212529870, "side_information_bytes": 13006, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 547.9008340043947, "doc_prefill_ms": 491.1570419790223, "compress_ms": 5.470708012580872, "serialize_ms": 47.444583964534104, "load_ms": 4.824541974812746, "receiver_prefill_ms": 0.0, "query_prefill_ms": 73.48570798058063, "ttft_ms": 73.48570798058063, "decode_ms": 4921.614417107776}, "total_wall_ms": 5001.988791045733, "ttft_ms": 73.48570798058063, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6697304064, "backend_memory": {"mlx_active_bytes": 7103706120, "mlx_peak_bytes": 7162344676, "mlx_cache_bytes": 7737651801, "process_rss_bytes": 6697304064}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 7162344676, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "522bffe332074bcb796f108c4005f47eec780836ba66d0efb9b603d20188ccef", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
87 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "74d9df9a04f363079fbb80ec", "attempt_id": "e3be8f7bcc14", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:28.490509Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6873248776}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "acd4936815db7cad", "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 212516864, "serialized_payload_bytes": 212529898, "side_information_bytes": 13034, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 373.191750026308, "doc_prefill_ms": 491.1570419790223, "compress_ms": 224.6105830417946, "serialize_ms": 148.3124999795109, "load_ms": 52.63670801650733, "receiver_prefill_ms": 0.0, "query_prefill_ms": 69.78820904623717, "ttft_ms": 69.78820904623717, "decode_ms": 4938.05329198949}, "total_wall_ms": 5062.4504160368815, "ttft_ms": 69.78820904623717, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 7312818176, "backend_memory": {"mlx_active_bytes": 7103542280, "mlx_peak_bytes": 7162180836, "mlx_cache_bytes": 7737815641, "process_rss_bytes": 7312818176}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 7162180836, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "6c67a21bc7e0c6e1c50b732378ea78ab585671684a96ad5d43a4f97a0218becf", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
88 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "0ebc2d3b95b910b895dba16b", "attempt_id": "edb994f3389f", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:33.691256Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6774158344}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 112899584, "serialized_payload_bytes": 112926716, "side_information_bytes": 27132, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 89.32725002523512, "doc_prefill_ms": 491.1570419790223, "compress_ms": 38.98474993184209, "serialize_ms": 50.01079197973013, "load_ms": 6.6009999718517065, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.833958046510816, "ttft_ms": 59.833958046510816, "decode_ms": 4563.0839159712195}, "total_wall_ms": 4631.997165968642, "ttft_ms": 59.833958046510816, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 109}, "provider_cached_usage": null, "process_memory": 6839533568, "backend_memory": {"mlx_active_bytes": 6897103880, "mlx_peak_bytes": 6978903204, "mlx_cache_bytes": 9336604729, "process_rss_bytes": 6839533568}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6978903204, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n revenue = r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n total += revenue\n tax = total * 0.09975\n total += tax\n return round(total, 2)", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e7a89df0990ce3437929cd0e918adccbe4bf4185ee218535531bb40e7ff755a6", "output_hash": "6a5767437ed79776be33a6ac25bf033ba327c6bfa72d04299e74247d9629c624"}
89 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2e3064455744be4e77fe8483", "attempt_id": "c03ae7602083", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:38.493748Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6720943112}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 59770368, "serialized_payload_bytes": 59797425, "side_information_bytes": 27057, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 122.98837502021343, "doc_prefill_ms": 491.1570419790223, "compress_ms": 90.87250009179115, "serialize_ms": 31.734207994304597, "load_ms": 4.436874995008111, "receiver_prefill_ms": 0.0, "query_prefill_ms": 45.52912502549589, "ttft_ms": 45.52912502549589, "decode_ms": 4216.681290999986}, "total_wall_ms": 4269.118999945931, "ttft_ms": 45.52912502549589, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 108}, "provider_cached_usage": null, "process_memory": 6784548864, "backend_memory": {"mlx_active_bytes": 6786085896, "mlx_peak_bytes": 6859957796, "mlx_cache_bytes": 9791039545, "process_rss_bytes": 6784548864}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6859957796, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] == 'Sud' or r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n tax = total * 0.09975\n total += tax\n return round(total, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 1.0, "pass_rate": 1.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "a2b102a2f28b534d74058a7540f8b4289c47979a55e3bc77a025b8b925612e15", "output_hash": "b9fa353e4e3364bf240cf377ea132be87c593ad7da514a543129c39d1b5e887a"}
90 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "efb3addedafe4789379451f0", "attempt_id": "a52d9e1663a8", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:42.906098Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 1853, "keep": 463, "compress_ms": 54.579207906499505}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113427, "side_information_bytes": 12883, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 88.03220803383738, "doc_prefill_ms": 491.1570419790223, "compress_ms": 54.63979090563953, "serialize_ms": 33.18020899314433, "load_ms": 2.2533750161528587, "receiver_prefill_ms": 0.0, "query_prefill_ms": 37.718416075222194, "ttft_ms": 37.718416075222194, "decode_ms": 3739.06579206232}, "total_wall_ms": 3782.042583101429, "ttft_ms": 37.718416075222194, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 100}, "provider_cached_usage": null, "process_memory": 6756958208, "backend_memory": {"mlx_active_bytes": 6783333384, "mlx_peak_bytes": 6819935468, "mlx_cache_bytes": 10784106553, "process_rss_bytes": 6756958208}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6819935468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total_revenue = 0\n total_cost = 0\n for row in rows:\n if row['revenue'] is not None and row['cost'] is not None:\n total_revenue += row['revenue']\n total_cost += row['cost']\n if total_revenue > 0 and total_cost > 0:\n return total_revenue - total_cost\n else:\n return 0\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1fb443d7753647899bdc487cfa5d58121408e20220200eca14b96b0da3dfe5db", "output_hash": "2c3ad5303d8d1f45396dfca7af62ca9ec49b63f96cdb18c9eca1e0e6dc18ce23"}
91 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "9dd21bff5ab11791cb50d7a5", "attempt_id": "5423139c786b", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:46.848905Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 1853, "keep": 463, "compress_ms": 67.42537498939782}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113421, "side_information_bytes": 12877, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 112.62058292049915, "doc_prefill_ms": 491.1570419790223, "compress_ms": 67.4783339491114, "serialize_ms": 44.90229196380824, "load_ms": 1.1183340102434158, "receiver_prefill_ms": 0.0, "query_prefill_ms": 42.3420409206301, "ttft_ms": 42.3420409206301, "decode_ms": 11749.81549999211}, "total_wall_ms": 11798.366833943874, "ttft_ms": 42.3420409206301, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 320}, "provider_cached_usage": null, "process_memory": 6758465536, "backend_memory": {"mlx_active_bytes": 6809023496, "mlx_peak_bytes": 6833395092, "mlx_cache_bytes": 13406824505, "process_rss_bytes": 6758465536}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6833395092, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "d9510e09d0ca8eb3b75b9730b22636fac2776b0f1df7ae8a095c1d0f3b1b855b", "output_hash": "96a784ac557a03befea2a2bf3c2c86086644fca8c3acf17e4a7eb949027d72ef"}
92 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e43d7b02322007475f60a2f7", "attempt_id": "c1a0f1b8e31b", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:46:58.701555Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1853, "keep": 463, "window_tokens": 45, "compress_ms": 248.77916695550084}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113467, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 278.2090000109747, "doc_prefill_ms": 491.1570419790223, "compress_ms": 251.14658300299197, "serialize_ms": 26.764166075736284, "load_ms": 11.841917061246932, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.18933392968029, "ttft_ms": 59.18933392968029, "decode_ms": 2920.7867920631543}, "total_wall_ms": 3273.8528749905527, "ttft_ms": 59.18933392968029, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 79}, "provider_cached_usage": null, "process_memory": 6846595072, "backend_memory": {"mlx_active_bytes": 6781498376, "mlx_peak_bytes": 6819935468, "mlx_cache_bytes": 13700950177, "process_rss_bytes": 6846595072}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6819935468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] != 'Sud':\n if r['units'] >= 2:\n total += r['units'] * (r['revenue'] / 100.0) * 0.09\n return round(total, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "344418aab1ee0e165ccb40fdaea3ae28b2491712429aad0877b2b70e2671f9e4", "output_hash": "8b9932740a653d50bb9a06974b348eac63c0262c73f7ab3adf800acb851a30ab"}
93 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b20154cd293d097bae3a92a0", "attempt_id": "9a6f4f817e57", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:02.238812Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6713603080}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 1853, "keep": 463, "window_tokens": 19, "compress_ms": 187.52808298449963}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113467, "side_information_bytes": 12923, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 208.3799170795828, "doc_prefill_ms": 491.1570419790223, "compress_ms": 187.91816697921604, "serialize_ms": 20.24979202542454, "load_ms": 2.7195829898118973, "receiver_prefill_ms": 0.0, "query_prefill_ms": 34.48229201603681, "ttft_ms": 34.48229201603681, "decode_ms": 4416.312083951198}, "total_wall_ms": 4461.860541952774, "ttft_ms": 34.48229201603681, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 121}, "provider_cached_usage": null, "process_memory": 6904283136, "backend_memory": {"mlx_active_bytes": 6786085896, "mlx_peak_bytes": 6819935468, "mlx_cache_bytes": 13716859117, "process_rss_bytes": 6904283136}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6819935468, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "def net_revenue(rows):\n total = 0\n for row in rows:\n if row['region'] != 'Nord' and row['region'] != 'Sud':\n if not row['returned']:\n if row['shipped_days'] < 7:\n total += round((row['revenue'] - row['tax']) * 0.9, 2)\n else:\n total += round((row['revenue'] - row['tax']), 2)\n else:\n total += round(row['revenue'], 2)\n return total", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "KeyError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "344418aab1ee0e165ccb40fdaea3ae28b2491712429aad0877b2b70e2671f9e4", "output_hash": "cf7408abaf3fe928a524f5803f95e73959c0d9bcbfa9c73f0704bc5451e2d084"}
94 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e82238acdb8214dd547b6279", "attempt_id": "bd85d54b6619", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:06.754008Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6660387848}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 1853, "keep": 463, "window_tokens": 45, "compress_ms": 133.0997080076486}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 53100544, "serialized_payload_bytes": 53113430, "side_information_bytes": 12886, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 158.15033309627324, "doc_prefill_ms": 491.1570419790223, "compress_ms": 134.8713330226019, "serialize_ms": 23.0540829943493, "load_ms": 8.13025003299117, "receiver_prefill_ms": 0.0, "query_prefill_ms": 1112.2218330856413, "ttft_ms": 1112.2218330856413, "decode_ms": 11907.870208960958}, "total_wall_ms": 13189.942457946017, "ttft_ms": 1112.2218330856413, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 320}, "provider_cached_usage": null, "process_memory": 6585827328, "backend_memory": {"mlx_active_bytes": 6809187336, "mlx_peak_bytes": 6850172160, "mlx_cache_bytes": 14307397873, "process_rss_bytes": 6585827328}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6850172160, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] != 'South' and r['region'] != 'South East':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West' and r['region'] != 'South East':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West' and r['region'] != 'South East' and r['region'] != 'South West':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South West' and r['region'] != 'South East' and r['region'] != 'South West' and r['region'] != 'South East':\n if r['region'] != 'South' and r['region'] != 'South East' and r['region'] != '", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": "exec:SyntaxError"}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "ea03e9d2020e968c315b0fad6087e5c357cc8a44908d6c2fd52e276c88655e8b", "output_hash": "6b997410a720d0cf9c319e5fbf09df9bb348c7fa1c510c5166bedb92e152c749"}
95 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8b9af9604a2dcd74186091cb", "attempt_id": "bbf489b5d01b", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:20.183489Z", "task_family": "exec", "document_id": "exec_00101_2000", "document_hash": "c8bf1a72f33c0fc6f87af8c3029f12cbff489bcb95ae36398cd522af1811a74f", "question_id": "q00_exec", "split": "dev", "cluster_id": "exec_00101_2000", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6717502472}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 1853, "keep": 926, "compress_ms": 81.46916702389717, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 1793, "source_tokens_receiver": 1793, "query_tokens": 45, "message_tokens": 1853, "message_utf8_bytes": 7944, "cache_tensor_bytes": 56419328, "serialized_payload_bytes": 56446110, "side_information_bytes": 26782, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 181.39470904134214, "doc_prefill_ms": 491.1570419790223, "compress_ms": 127.8945830417797, "serialize_ms": 53.03558299783617, "load_ms": 3.911250038072467, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.55883290152997, "ttft_ms": 59.55883290152997, "decode_ms": 3817.130250041373}, "total_wall_ms": 3883.2200409378856, "ttft_ms": 59.55883290152997, "input_usage": {"receiver_prefix_tokens": 1853, "dense_cache_bytes": 212516864, "query_tokens": 45}, "output_usage": {"output_tokens": 96}, "provider_cached_usage": null, "process_memory": 6647808000, "backend_memory": {"mlx_active_bytes": 6786315272, "mlx_peak_bytes": 6845637732, "mlx_cache_bytes": 14337102065, "process_rss_bytes": 6647808000}, "peak_prefill_memory": 7118885336, "peak_decode_memory": 6845637732, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "```python\ndef net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['region'] != 'North America':\n continue\n if r['returned']:\n continue\n if r['units'] < 2:\n continue\n total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n return round(total, 2)\n```", "evidence_artifact": null, "reference_key_id": "keys/exec/exec_00101_2000.json", "task_scores": {"correct": 0.0, "pass_rate": 0.0, "error": null}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "854e85585cd5598ae69453b12fdffc0e8851d877afbd9e6047416972c1c33dd1", "output_hash": "55874831713d841ac59fbffaa1fb9d32a2ded0513cbc33751bbbec2c460f9881"}
96 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "44c9f24bdb157946f705579d", "attempt_id": "1167560c78f2", "run_group_id": "E0_smoke_llama:A0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:24.953633Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A0", "implementation_fidelity": "custom", "compression_config": {"realized": {"note": "receiver gets no source context"}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 0, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 82, "message_tokens": 0, "message_utf8_bytes": 0, "cache_tensor_bytes": null, "serialized_payload_bytes": 0, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.022249994799494743, "decode_payload_ms": 0.0, "receiver_prefill_ms": 64.43300005048513, "ttft_ms": 39.59012497216463, "decode_ms": 90.72749991901219}, "total_wall_ms": 195.6150830956176, "ttft_ms": 39.59012497216463, "input_usage": {"receiver_prompt_tokens": 82}, "output_usage": {"output_tokens": 2}, "provider_cached_usage": null, "process_memory": 6661668864, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6510474664, "mlx_cache_bytes": 379634332, "process_rss_bytes": 6661668864}, "peak_prefill_memory": null, "peak_decode_memory": 6510474664, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Unknown.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "output_hash": "9cf9f50925b2a7803c73826a305235d4215dc428721c731072a9f2c0f5d6ae5a"}
97 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "77089f1b85d697b839c288d5", "attempt_id": "0f1eb80e125d", "run_group_id": "E0_smoke_llama:A1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:25.199886Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A1", "implementation_fidelity": "custom", "compression_config": {}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 3257, "message_tokens": 3165, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 15217, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.013249926269054413, "decode_payload_ms": 0.0, "receiver_prefill_ms": 926.2564160162583, "ttft_ms": 46.42666596919298, "decode_ms": 396.2914999574423}, "total_wall_ms": 1375.4641669802368, "ttft_ms": 46.42666596919298, "input_usage": {"receiver_prompt_tokens": 3257}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6662848512, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7252325440, "mlx_cache_bytes": 1671217820, "process_rss_bytes": 6662848512}, "peak_prefill_memory": null, "peak_decode_memory": 7252325440, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
98 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d69c345c47e7d7fba1448da5", "attempt_id": "3e503b976551", "run_group_id": "E0_smoke_llama:A2|budget_tokens=512", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:26.604964Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A2", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "realized": {"budget_tokens": 512, "source_tokens": 3165}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 613, "message_tokens": 516, "message_utf8_bytes": 2753, "cache_tensor_bytes": null, "serialized_payload_bytes": 2753, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 4.279125016182661, "decode_payload_ms": 0.0, "receiver_prefill_ms": 159.79537495877594, "ttft_ms": 39.40404194872826, "decode_ms": 145.06600005552173}, "total_wall_ms": 346.0776249412447, "ttft_ms": 39.40404194872826, "input_usage": {"receiver_prompt_tokens": 613}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": 6664994816, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6865649128, "mlx_cache_bytes": 2124532780, "process_rss_bytes": 6664994816}, "peak_prefill_memory": null, "peak_decode_memory": 6865649128, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition approach.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "b4ebdc3f71e0de8c27d74d12c7425f1cc913cf419027bf0aa9d66ba8e1fb4ef4", "output_hash": "c91ced75830c795910278259ba26b352cb3b2bb8bcb4ece33a579acee947dd98"}
99 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "885c9fa0620152363db82124", "attempt_id": "3a29092e34ab", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:26.979472Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "realized_tokens": 502, "n_sentences": 13}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 595, "message_tokens": 502, "message_utf8_bytes": 2268, "cache_tensor_bytes": null, "serialized_payload_bytes": 2268, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 4.544583964161575, "decode_payload_ms": 0.0, "receiver_prefill_ms": 175.96541601233184, "ttft_ms": 17.405707971192896, "decode_ms": 79.07637499738485}, "total_wall_ms": 274.16608296334743, "ttft_ms": 17.405707971192896, "input_usage": {"receiver_prompt_tokens": 595}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 6665322496, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6855114216, "mlx_cache_bytes": 2478935084, "process_rss_bytes": 6665322496}, "peak_prefill_memory": null, "peak_decode_memory": 6855114216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "179bd6082f22e64b2bb3b7a0fb33d014d40dd0c0a2527b41e66dc1a8cb97a1b6", "output_hash": "65cb98a4671af923622f6d04b76c9ce50b86b153894574e55a962cb4f727b91f"}
100 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "e9f57946fcf42b900e80e15d", "attempt_id": "1e532de867fd", "run_group_id": "E0_smoke_llama:A3|budget_tokens=512|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:27.278586Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A3", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "aware", "realized": {"budget_tokens": 512, "realized_tokens": 500, "n_sentences": 23}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 594, "message_tokens": 501, "message_utf8_bytes": 2495, "cache_tensor_bytes": null, "serialized_payload_bytes": 2495, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 1.601582975126803, "decode_payload_ms": 0.0, "receiver_prefill_ms": 174.43599994294345, "ttft_ms": 37.115958984941244, "decode_ms": 280.2286660298705}, "total_wall_ms": 494.99341601040214, "ttft_ms": 37.115958984941244, "input_usage": {"receiver_prompt_tokens": 594}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6665338880, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6855114216, "mlx_cache_bytes": 2478935084, "process_rss_bytes": 6665338880}, "peak_prefill_memory": null, "peak_decode_memory": 6855114216, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4615384615384615, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "785023803d7f2c38c80debcdc8309ffa1c94e3c99227c52a36e7e66d830d2d9e", "output_hash": "62be2a82b9b81284ad4ad3e5bf7aa56d7866450a753eeed5461895891c88b2e9"}
101 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "d65b20dcb306c0140c6cbee5", "attempt_id": "9a679b545f77", "run_group_id": "E0_smoke_llama:A4|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:47:49.999549Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 609, "message_tokens": 512, "message_utf8_bytes": 2544, "cache_tensor_bytes": null, "serialized_payload_bytes": 2544, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22201.99812494684, "decode_payload_ms": 0.0, "receiver_prefill_ms": 163.27533405274153, "ttft_ms": 33.47712499089539, "decode_ms": 271.6693750116974}, "total_wall_ms": 470.18758300691843, "ttft_ms": 33.47712499089539, "input_usage": {"receiver_prompt_tokens": 609, "sender_input_tokens": 3299, "sender_output_tokens": 512, "sender_prefill_ms": 847.9583340231329, "sender_ttft_ms": 43.58004208188504, "sender_decode_ms": 21307.24516592454}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 6666305536, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6863125992, "mlx_cache_bytes": 4080176172, "process_rss_bytes": 6666305536}, "peak_prefill_memory": null, "peak_decode_memory": 6863125992, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4615384615384615, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "a92a3840adade198f1f17e7bef7cf09fa2fc2a50f1fe33c73523c5400c06ace4", "output_hash": "62be2a82b9b81284ad4ad3e5bf7aa56d7866450a753eeed5461895891c88b2e9"}
102 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "6bda13bb6dac6b19c1f79a9b", "attempt_id": "7752cbe610ee", "run_group_id": "E0_smoke_llama:A5|budget_tokens=512|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:12.350377Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "realized": {"budget_tokens": 512, "n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 488, "message_tokens": 380, "message_utf8_bytes": 1257, "cache_tensor_bytes": null, "serialized_payload_bytes": 1257, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 21855.362500064075, "decode_payload_ms": 0.0, "receiver_prefill_ms": 165.4428340261802, "ttft_ms": 23.285583942197263, "decode_ms": 217.3186249565333}, "total_wall_ms": 407.3195829987526, "ttft_ms": 23.285583942197263, "input_usage": {"receiver_prompt_tokens": 488, "sender_input_tokens": 3329, "sender_output_tokens": 512, "sender_prefill_ms": 949.9937500804663, "sender_ttft_ms": 55.828875047154725, "sender_decode_ms": 20846.271000104025}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 6667468800, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6764788824, "mlx_cache_bytes": 4628761644, "process_rss_bytes": 6667468800}, "peak_prefill_memory": null, "peak_decode_memory": 6764788824, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD model", "evidence_artifact": {"n_facts": 9, "unresolved_quotes": 0, "parse_ok": true}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "8872c1ee69cdc942cca378c38034ceb53ada658177b2b563192861748b58062e", "output_hash": "158b135e4f20814888b9161908fdd8705a04091174ac37b666dd11ae3cc5a4a7"}
103 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "93d51aaa500866412d07fe2f", "attempt_id": "d97b679051fc", "run_group_id": "E0_smoke_llama:A6|budget_tokens=512|max_fetch_bytes=2000|max_lookups=6|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:34.927724Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A6", "implementation_fidelity": "custom", "compression_config": {"budget_tokens": 512, "access": "blind", "max_fetch_bytes": 2000, "max_lookups": 6, "realized": {"n_refs": 9}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 858, "message_tokens": 765, "message_utf8_bytes": 830, "cache_tensor_bytes": null, "serialized_payload_bytes": 830, "side_information_bytes": 0, "fetched_bytes": 1660, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 22146.301458007656, "decode_payload_ms": 0.0, "receiver_prefill_ms": 227.50466701108962, "ttft_ms": 43.446708004921675, "decode_ms": 36.40404099132866}, "total_wall_ms": 309.42874995525926, "ttft_ms": 43.446708004921675, "input_usage": {"receiver_prompt_tokens": 858, "sender_input_tokens": 3329, "sender_output_tokens": 512, "sender_prefill_ms": 905.3275000769645, "sender_ttft_ms": 68.31662508193403, "sender_decode_ms": 21169.65887497645}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": 6667829248, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 6933646424, "mlx_cache_bytes": 5173365804, "process_rss_bytes": 6667829248}, "peak_prefill_memory": null, "peak_decode_memory": 6933646424, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"lookups": 6, "fetched_bytes": 1660, "retrieved_refs": 6}, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "93bbce05a459c3fb41371ce4ff355223722554545cc4dac340f74b2cd6065f19", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
104 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "c1a6e4b7f5c4a44097a4f3f4", "attempt_id": "9c9d37ae535b", "run_group_id": "E0_smoke_llama:A7|codec=zstd", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:35.260676Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6425506824}, "weight_precision": "bf16", "method_id": "A7", "implementation_fidelity": "custom", "compression_config": {"codec": "zstd", "realized": {"codec": "zstd", "ratio": 2.8798258894776683}}, "question_access": "none", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 3257, "message_tokens": 3165, "message_utf8_bytes": 15217, "cache_tensor_bytes": null, "serialized_payload_bytes": 5284, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 0.15029101632535458, "decode_payload_ms": 0.028749927878379822, "receiver_prefill_ms": 884.1438330709934, "ttft_ms": 55.560875101946294, "decode_ms": 462.33312506228685}, "total_wall_ms": 1407.4600409949198, "ttft_ms": 55.560875101946294, "input_usage": {"receiver_prompt_tokens": 3257}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 6668009472, "backend_memory": {"mlx_active_bytes": 6425506824, "mlx_peak_bytes": 7252325440, "mlx_cache_bytes": 5173365804, "process_rss_bytes": 6668009472}, "peak_prefill_memory": null, "peak_decode_memory": 7252325440, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
105 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "abd8f72d5ebf8d2a9042c92c", "attempt_id": "f44c40de4bc9", "run_group_id": "E0_smoke_llama:B0", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:37.563174Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 7177860104}, "weight_precision": "bf16", "method_id": "B0", "implementation_fidelity": "custom", "compression_config": {"realized": {}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 369868800, "serialized_payload_bytes": 369881830, "side_information_bytes": 13030, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 872.6028749952093, "doc_prefill_ms": 768.3387080905959, "compress_ms": 8.387916954234242, "serialize_ms": 90.83100000862032, "load_ms": 7.476166938431561, "receiver_prefill_ms": 0.0, "query_prefill_ms": 65.79979206435382, "ttft_ms": 65.79979206435382, "decode_ms": 491.8956670444459}, "total_wall_ms": 567.7750420290977, "ttft_ms": 65.79979206435382, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 7408992256, "backend_memory": {"mlx_active_bytes": 7553119240, "mlx_peak_bytes": 7646950584, "mlx_cache_bytes": 6101340461, "process_rss_bytes": 7408992256}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7646950584, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "8ee5cf8383bacbbd8af5e34008644b55e21053faeefa305d0b6c29ed3b5977a7", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
106 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "fde7a2e940724dc99dfc4dcd", "attempt_id": "41e64241eecd", "run_group_id": "E0_smoke_llama:B1", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:38.273477Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 7177860104}, "weight_precision": "bf16", "method_id": "B1", "implementation_fidelity": "custom", "compression_config": {"realized": {"serialize_roundtrip": true}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": "95737b617c283ebf", "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 369868800, "serialized_payload_bytes": 369881858, "side_information_bytes": 13058, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 117.73079202976078, "doc_prefill_ms": 768.3387080905959, "compress_ms": 7.5602500000968575, "serialize_ms": 109.97679200954735, "load_ms": 49.44625007919967, "receiver_prefill_ms": 0.0, "query_prefill_ms": 58.2056250423193, "ttft_ms": 58.2056250423193, "decode_ms": 428.46745892893523}, "total_wall_ms": 538.6375420494005, "ttft_ms": 58.2056250423193, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 7779844096, "backend_memory": {"mlx_active_bytes": 7553119240, "mlx_peak_bytes": 7646950584, "mlx_cache_bytes": 6101340461, "process_rss_bytes": 7779844096}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7646950584, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "06d4d3fe4804f01b770c9587580d9b01847dda9cbf57b892fc812328806ca6c5", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
107 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "b9564e291f64162d9ab2f97e", "attempt_id": "d5b9da4adb6f", "run_group_id": "E0_smoke_llama:B2|bits=8", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:38.939579Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 7005369352}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 8, "realized": {"bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 196492800, "serialized_payload_bytes": 196520056, "side_information_bytes": 27256, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 103.859500028193, "doc_prefill_ms": 768.3387080905959, "compress_ms": 52.26229201070964, "serialize_ms": 51.22220900375396, "load_ms": 6.104291067458689, "receiver_prefill_ms": 0.0, "query_prefill_ms": 175.92745902948081, "ttft_ms": 175.92745902948081, "decode_ms": 421.5575830312446}, "total_wall_ms": 606.0640841023996, "ttft_ms": 175.92745902948081, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 7977893888, "backend_memory": {"mlx_active_bytes": 7206302728, "mlx_peak_bytes": 7295243472, "mlx_cache_bytes": 7318172845, "process_rss_bytes": 7977893888}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7295243472, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "917cb67928b6971fc0158a2501ad4fe2a4db28dc15dec6a692c9b208d7a1f029", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
108 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "2061cab6ef742a32c401c02b", "attempt_id": "e0985f83fa6c", "run_group_id": "E0_smoke_llama:B2|bits=4", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:39.662899Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6912701448}, "weight_precision": "bf16", "method_id": "B2", "implementation_fidelity": "custom", "compression_config": {"bits": 4, "realized": {"bits": 4}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 104025600, "serialized_payload_bytes": 104052706, "side_information_bytes": 27106, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 91.90604102332145, "doc_prefill_ms": 768.3387080905959, "compress_ms": 50.19495799206197, "serialize_ms": 41.365165961906314, "load_ms": 4.855667008087039, "receiver_prefill_ms": 0.0, "query_prefill_ms": 145.75187501031905, "ttft_ms": 145.75187501031905, "decode_ms": 435.59229106176645}, "total_wall_ms": 588.821041979827, "ttft_ms": 145.75187501031905, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 8187625472, "backend_memory": {"mlx_active_bytes": 7020049416, "mlx_peak_bytes": 7101129960, "mlx_cache_bytes": 7934092589, "process_rss_bytes": 8187625472}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7101129960, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "089268d279957f99a33245ea777c87a7dfb5186fb242114d885d00081725b7d2", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
109 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "4d3758fd158d22e9937a76e5", "attempt_id": "d588d2ad483d", "run_group_id": "E0_smoke_llama:B3|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:40.340883Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6899856392}, "weight_precision": "bf16", "method_id": "B3", "implementation_fidelity": "inspired", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "sink_recent", "keep_frac": 0.25, "T": 3225, "keep": 806, "compress_ms": 27.757166884839535}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451421, "side_information_bytes": 12893, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 64.23241703305393, "doc_prefill_ms": 768.3387080905959, "compress_ms": 27.796707930974662, "serialize_ms": 36.2354579847306, "load_ms": 1.9377500284463167, "receiver_prefill_ms": 0.0, "query_prefill_ms": 35.60299996752292, "ttft_ms": 35.60299996752292, "decode_ms": 138.23416701052338}, "total_wall_ms": 179.83395792543888, "ttft_ms": 35.60299996752292, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 8184594432, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8174708013, "process_rss_bytes": 8184594432}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition technique", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "cfd45f7f52bfb277d73e198d2a330de3371aa41aec2f77f0731c89ea088e7299", "output_hash": "adc705b949def2e4833f11eabfdb5cc131f5c7801bcaf695f325f05dadc1903f"}
110 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3c1b141113418107b8e9cf68", "attempt_id": "84d644ccfa36", "run_group_id": "E0_smoke_llama:B4|keep_frac=0.25", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:40.618881Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6899856392}, "weight_precision": "bf16", "method_id": "B4", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.25, "realized": {"policy": "random", "keep_frac": 0.25, "T": 3225, "keep": 806, "compress_ms": 29.710708069615066}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451416, "side_information_bytes": 12888, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 71.83591590728611, "doc_prefill_ms": 768.3387080905959, "compress_ms": 29.746290994808078, "serialize_ms": 41.88670904841274, "load_ms": 3.7346669705584645, "receiver_prefill_ms": 0.0, "query_prefill_ms": 49.082000041380525, "ttft_ms": 49.082000041380525, "decode_ms": 202.58479204494506}, "total_wall_ms": 257.95587501488626, "ttft_ms": 49.082000041380525, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": 8278245376, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8199021869, "process_rss_bytes": 8278245376}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Extension of NetVLAD", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4615384615384615, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "bfc5aad5ad47b143ff2ae900af44295c81902b72194b44a72a356052e358fbfb", "output_hash": "8f140fbbbd872bb79f452c7a3e155d08de9d9b3df5f5ad753e80b4846eb8431c"}
111 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "ef70aa1aacd6d3a3581298dd", "attempt_id": "c83874a71bb5", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:40.903176Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6807188488}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3225, "keep": 806, "window_tokens": 32, "compress_ms": 166.13045800477266}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451461, "side_information_bytes": 12933, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 196.04720908682793, "doc_prefill_ms": 768.3387080905959, "compress_ms": 168.97325008176267, "serialize_ms": 26.782125001773238, "load_ms": 1.180332968942821, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.43370796740055, "ttft_ms": 59.43370796740055, "decode_ms": 406.5587919903919}, "total_wall_ms": 666.2564580328763, "ttft_ms": 59.43370796740055, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": 8287830016, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8623891861, "process_rss_bytes": 8287830016}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of NetVLAD for face recognition.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.37499999999999994, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "abd1bbad98ec9628f9d26690ae31cf6986bf28f92f185dd733e97b813a4d72f5", "output_hash": "9816a4414797f3bf80cbc9ca7be903c241ad826f66b9aad1df5d1829dbe8b0b9"}
112 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "3ad4f21970c0d4d1b502d40a", "attempt_id": "8d6d9f43c4d7", "run_group_id": "E0_smoke_llama:B5|keep_frac=0.25|blind", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:41.752781Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6899856392}, "weight_precision": "bf16", "method_id": "B5", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "blind", "realized": {"policy": "attention", "keep_frac": 0.25, "T": 3225, "keep": 806, "window_tokens": 19, "compress_ms": 137.13216700125486}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451461, "side_information_bytes": 12933, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 158.06108398828655, "doc_prefill_ms": 768.3387080905959, "compress_ms": 137.29495892766863, "serialize_ms": 20.584333105944097, "load_ms": 1.8588330131024122, "receiver_prefill_ms": 0.0, "query_prefill_ms": 47.956917085684836, "ttft_ms": 47.956917085684836, "decode_ms": 247.80429096426815}, "total_wall_ms": 300.38979195524007, "ttft_ms": 47.956917085684836, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": 8287911936, "backend_memory": {"mlx_active_bytes": 6997111816, "mlx_peak_bytes": 7041660088, "mlx_cache_bytes": 8646894997, "process_rss_bytes": 8287911936}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7041660088, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Ghost-VLAD pooling method.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "abd1bbad98ec9628f9d26690ae31cf6986bf28f92f185dd733e97b813a4d72f5", "output_hash": "bb4768fc890447a743a28d413f4b9165fc5b5887a6b9e7f30a3648c31602aa4e"}
113 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "172f04f7b6a5ab7ec4d24438", "attempt_id": "db4912d84e35", "run_group_id": "E0_smoke_llama:B6|keep_frac=0.25|aware", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:42.082088Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6807188488}, "weight_precision": "bf16", "method_id": "B6", "implementation_fidelity": "faithful_reimplementation", "compression_config": {"keep_frac": 0.25, "access": "aware", "realized": {"policy": "pyramid", "keep_frac": 0.25, "T": 3225, "keep": 806, "window_tokens": 32, "compress_ms": 164.159334031865}}, "question_access": "aware", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 92438528, "serialized_payload_bytes": 92451489, "side_information_bytes": 12961, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 193.5198330320418, "doc_prefill_ms": 768.3387080905959, "compress_ms": 166.54054203536361, "serialize_ms": 26.766416965983808, "load_ms": 11.041625053621829, "receiver_prefill_ms": 0.0, "query_prefill_ms": 1063.2581249810755, "ttft_ms": 1063.2581249810755, "decode_ms": 370.9792080335319}, "total_wall_ms": 1642.524458002299, "ttft_ms": 1063.2581249810755, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": 7598637056, "backend_memory": {"mlx_active_bytes": 6997996552, "mlx_peak_bytes": 7121915044, "mlx_cache_bytes": 10613161933, "process_rss_bytes": 7598637056}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7121915044, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "An extension of the NetVLAD approach.", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.4285714285714285, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "c839577fd550993df17a39b7857d9ea072aa21bed74cdcc84d8e1230292586f1", "output_hash": "35066d4391427300133fbec39fa550f39d33b0cf8700f5300bcb8b4c8f852d68"}
114 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E0_smoke_llama", "trial_id": "8cddeb96fd95a16ceee43ed2", "attempt_id": "d9a564f246f6", "run_group_id": "E0_smoke_llama:B9|bits=8|keep_frac=0.5", "protocol_hash": "unfrozen", "config_hash": "c97f7990508e79bc75587da79eb05cc9fc689fb427f68881c11b47633ac78b50", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:48:43.936211Z", "task_family": "public_qasper", "document_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "document_hash": "1704f1d50b2554aebd4feef979b444ef2634aca15e51d3608364aaf7b146ae43", "question_id": "q00", "split": "dev", "cluster_id": "lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084", "sender_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"sender": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763", "receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {"mlx_active_before": 6907196424}, "weight_precision": "bf16", "method_id": "B9", "implementation_fidelity": "custom", "compression_config": {"keep_frac": 0.5, "bits": 8, "realized": {"policy": "sink_recent", "keep_frac": 0.5, "T": 3225, "keep": 1612, "compress_ms": 68.63458303268999, "bits": 8}}, "question_access": "blind", "topology": "single_handoff", "handoff_index": 1, "reuse_count": 1, "cache_hit": true, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": 3165, "source_tokens_receiver": 3165, "query_tokens": 32, "message_tokens": 3225, "message_utf8_bytes": 15217, "cache_tensor_bytes": 98215936, "serialized_payload_bytes": 98243143, "side_information_bytes": 27207, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"encode_ms": 183.25416697189212, "doc_prefill_ms": 768.3387080905959, "compress_ms": 120.4705840209499, "serialize_ms": 62.467875075526536, "load_ms": 8.85850004851818, "receiver_prefill_ms": 0.0, "query_prefill_ms": 59.84200001694262, "ttft_ms": 59.84200001694262, "decode_ms": 125.22991700097919}, "total_wall_ms": 196.51483302004635, "ttft_ms": 59.84200001694262, "input_usage": {"receiver_prefix_tokens": 3225, "dense_cache_bytes": 369868800, "query_tokens": 32}, "output_usage": {"output_tokens": 3}, "provider_cached_usage": null, "process_memory": 7699415040, "backend_memory": {"mlx_active_bytes": 7009039368, "mlx_peak_bytes": 7073338576, "mlx_cache_bytes": 10619427069, "process_rss_bytes": 7699415040}, "peak_prefill_memory": 7242626112, "peak_decode_memory": 7073338576, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Face recognition technique", "evidence_artifact": null, "reference_key_id": "keys/public_qasper/lb_qasper_db6c4d611b78c840972b57ee0b245003566b982feaff1084.json", "task_scores": {"f1": 0.0, "em": 0.0, "abstained": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E0_smoke_llama.log", "payload_hash": "74c04b99846bb7fc6b88c66a3c6d2a9cba3987192872977c7c90b213d45ae0f0", "output_hash": "adc705b949def2e4833f11eabfdb5cc131f5c7801bcaf695f325f05dadc1903f"}
added runs/raw/E4_smoke.jsonl +60 −0
@@ -0,0 +1,60 @@
1 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e4d6014a524a26cf323d29f6", "attempt_id": "198debfc8cf2", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:05.711305Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 70.56966691743582, "ttft_ms": 13.580082915723324, "decode_ms": 133.09933303389698}, "total_wall_ms": 7606.1354170087725, "ttft_ms": 13.580082915723324, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
2 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3a7862e06d5a28621d6558c3", "attempt_id": "ea464fda49ef", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:05.947390Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 70.54591609630734, "ttft_ms": 13.318874989636242, "decode_ms": 13.681416981853545}, "total_wall_ms": 7486.423374968581, "ttft_ms": 13.318874989636242, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
3 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "8fe0656fc337780a8dd4109b", "attempt_id": "bc2a8af8de0e", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.061895Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 70.04433299880475, "ttft_ms": 13.816624996252358, "decode_ms": 13.376791961491108}, "total_wall_ms": 7486.115916981362, "ttft_ms": 13.816624996252358, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
4 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "d6c66202a9977e9ce9be9b6e", "attempt_id": "df2eb4992ee4", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.175911Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 72.721833945252, "ttft_ms": 13.34204210434109, "decode_ms": 13.484457973390818}, "total_wall_ms": 7488.403292139992, "ttft_ms": 13.34204210434109, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
5 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f9daf53ae4c359f1ffcfe01b", "attempt_id": "e05c9dd05f08", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.292234Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 605, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 80.6656670756638, "ttft_ms": 13.283834094181657, "decode_ms": 119.9960419908166}, "total_wall_ms": 7602.7952920412645, "ttft_ms": 13.283834094181657, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
6 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "5c93c75e9fbdbcaa62c929d5", "attempt_id": "7d85efdf21da", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.521949Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 603, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 83.6761670652777, "ttft_ms": 13.48012499511242, "decode_ms": 133.4958749357611}, "total_wall_ms": 7619.5156670873985, "ttft_ms": 13.48012499511242, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
7 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "0d1c21da0c2ef7ac459880a8", "attempt_id": "3da528f59695", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.768717Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 82.41020806599408, "ttft_ms": 13.700834009796381, "decode_ms": 13.601665967144072}, "total_wall_ms": 7498.5860420856625, "ttft_ms": 13.700834009796381, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "no", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "9390298f3fb0c5b160498935d79cb139aef28e1c47358b4bbba61862b9c26e59"}
8 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "d5baf7618f6c26d9cb7e6e8e", "attempt_id": "b14008ba6e2c", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:06.895344Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 595, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 87.12095802184194, "ttft_ms": 13.858709018677473, "decode_ms": 13.865667046047747}, "total_wall_ms": 7503.698958083987, "ttft_ms": 13.858709018677473, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
9 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "7a08e0ff0827c646feb5a691", "attempt_id": "c40dbba05788", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.027877Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 611, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 92.70949999336153, "ttft_ms": 13.947332976385951, "decode_ms": 13.76683299895376}, "total_wall_ms": 7509.262875071727, "ttft_ms": 13.947332976385951, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
10 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "715da87439b9c2da1d80846c", "attempt_id": "0288e31dbdee", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.164798Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 603, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 88.89854198787361, "ttft_ms": 13.895749929361045, "decode_ms": 13.664584024809301}, "total_wall_ms": 7505.350458086468, "ttft_ms": 13.895749929361045, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
11 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f670c6c1f9a3016f46e0668a", "attempt_id": "f3db058d1537", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.297840Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 612, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 94.91300000809133, "ttft_ms": 13.864832930266857, "decode_ms": 68.32533294800669}, "total_wall_ms": 7565.9649590961635, "ttft_ms": 13.864832930266857, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
12 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "106680d7384b75e470629b46", "attempt_id": "ff2215fa3b74", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.490963Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 607, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 83.91812501940876, "ttft_ms": 13.685125042684376, "decode_ms": 96.26008395571262}, "total_wall_ms": 7582.891375059262, "ttft_ms": 13.685125042684376, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1379172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "7d9e6c79de4e797c6177e40b49a810a6363041af98982dbd4243baf4bec5c971"}
13 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "69c2449a5a3796644cc816ca", "attempt_id": "730ec68711a2", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.700942Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 602, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 80.60191699769348, "ttft_ms": 13.449500082060695, "decode_ms": 55.076666991226375}, "total_wall_ms": 7538.049542112276, "ttft_ms": 13.449500082060695, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7640", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b7aa5cfd1d2529920befa138d570102a53952edfdd4823a958212feea0698adf"}
14 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "20e3cca49e53c7a7a4486d47", "attempt_id": "be3fa64e72af", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.866998Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 607, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 79.90162493661046, "ttft_ms": 13.807334005832672, "decode_ms": 13.882457977160811}, "total_wall_ms": 7496.488250093535, "ttft_ms": 13.807334005832672, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
15 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "93cddd2a9c80209579d4603d", "attempt_id": "960b65285402", "run_group_id": "E4_smoke:A4|closed|qwen3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:07.989966Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 608, "message_tokens": 512, "message_utf8_bytes": 1196, "cache_tensor_bytes": null, "serialized_payload_bytes": 1196, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7388.310500071384, "hops": {"1": 7388.310500071384}, "receiver_prefill_ms": 80.64408297650516, "ttft_ms": 13.768291915766895, "decode_ms": 13.53445800486952}, "total_wall_ms": 7496.865458087996, "ttft_ms": 13.768291915766895, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "f811e3d2b4b96fb92f1cc9651f50294bf8a18ad7c33cafac648e5969f236fb1e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
16 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "fed4aa61963a7bf8327a9044", "attempt_id": "e294c1d78b77", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.145297Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 591, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.30716701410711, "ttft_ms": 13.295792043209076, "decode_ms": 133.86049994733185}, "total_wall_ms": 14635.955083067529, "ttft_ms": 13.295792043209076, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WG-14H-1182", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "e454fec67e299085db563490eff5a43753bbeed6d185376c1a5962c124db19eb"}
17 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "1fd962b7fed8b7be0f957943", "attempt_id": "a185532acd87", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.379601Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.43875004071742, "ttft_ms": 13.473832979798317, "decode_ms": 13.634915929287672}, "total_wall_ms": 14514.995125005953, "ttft_ms": 13.473832979798317, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
18 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "44736994ac7d3c6e955ca246", "attempt_id": "1ba1eb8cf9e4", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.492830Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 597, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.44708398077637, "ttft_ms": 13.740207999944687, "decode_ms": 13.40095791965723}, "total_wall_ms": 14515.074791037478, "ttft_ms": 13.740207999944687, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
19 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3b4059272a0084c78eb7bb71", "attempt_id": "ee0abde30be1", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.605864Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.15895803831518, "ttft_ms": 13.322583981789649, "decode_ms": 13.661875040270388}, "total_wall_ms": 14514.662832953036, "ttft_ms": 13.322583981789649, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
20 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "a5a377217f065df885e17778", "attempt_id": "eab1d937151f", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.718152Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 602, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.07291703484952, "ttft_ms": 13.24529096018523, "decode_ms": 120.1137499883771}, "total_wall_ms": 14621.930750086904, "ttft_ms": 13.24529096018523, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
21 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "75305a0fa99138dc19870aeb", "attempt_id": "1e9b18d736a8", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:15.937819Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.9936660239473, "ttft_ms": 13.173334067687392, "decode_ms": 133.35729099344462}, "total_wall_ms": 14635.019749985076, "ttft_ms": 13.173334067687392, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
22 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "02cb774895cb7f4200e25ac7", "attempt_id": "f91932e6db80", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.171074Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 594, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.08262509386986, "ttft_ms": 13.466249918565154, "decode_ms": 13.275083969347179}, "total_wall_ms": 14514.30133311078, "ttft_ms": 13.466249918565154, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "no", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "9390298f3fb0c5b160498935d79cb139aef28e1c47358b4bbba61862b9c26e59"}
23 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "836a2db482e48950e8112ef4", "attempt_id": "3ec99045954c", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.283280Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 592, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.89300006534904, "ttft_ms": 13.15629098098725, "decode_ms": 13.341167010366917}, "total_wall_ms": 14514.862333075143, "ttft_ms": 13.15629098098725, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
24 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e1d9c69d659fac9dbfa05d73", "attempt_id": "0581ed045141", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.396005Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 608, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.83249995391816, "ttft_ms": 13.473790953867137, "decode_ms": 13.576042023487389}, "total_wall_ms": 14515.38841612637, "ttft_ms": 13.473790953867137, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
25 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "590e8690bc20f0eeb26cf3fe", "attempt_id": "1404cc3d5f3f", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.508754Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 600, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 69.97666589450091, "ttft_ms": 13.296290999278426, "decode_ms": 13.450499973259866}, "total_wall_ms": 14515.232374076732, "ttft_ms": 13.296290999278426, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
26 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "acfb0023ca11a1d053425fb0", "attempt_id": "97c16ca89c34", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.622042Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 609, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.51529199816287, "ttft_ms": 13.40104197151959, "decode_ms": 66.48116698488593}, "total_wall_ms": 14568.905583000742, "ttft_ms": 13.40104197151959, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
27 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3ed2e99bfe065e96ba5567e6", "attempt_id": "1f628ef9ac35", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:16.788509Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 604, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.03845798317343, "ttft_ms": 13.494165963493288, "decode_ms": 133.29291704576463}, "total_wall_ms": 14635.345748974942, "ttft_ms": 13.494165963493288, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "19,431,432", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "06fe08c35e6644a2505156a3386f0e4d98a25b828e2efd638d491d710f856103"}
28 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f6d8b0874f8a14cbf833d123", "attempt_id": "85a8507f0396", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:17.021966Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 599, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.00924996100366, "ttft_ms": 13.551250100135803, "decode_ms": 52.76383401360363}, "total_wall_ms": 14554.786042077467, "ttft_ms": 13.551250100135803, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 4}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7640", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b7aa5cfd1d2529920befa138d570102a53952edfdd4823a958212feea0698adf"}
29 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3bbb5e2647d0d2231c6dc1a3", "attempt_id": "fa24afedd8b2", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:17.174862Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 604, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 70.16887504141778, "ttft_ms": 13.312499970197678, "decode_ms": 13.794707949273288}, "total_wall_ms": 14515.795915969647, "ttft_ms": 13.312499970197678, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
30 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "f81cbc0cf864774a3798a98d", "attempt_id": "40ab50e944d0", "run_group_id": "E4_smoke:A4|closed|qwen3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:17.288905Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A4", "implementation_fidelity": "custom", "compression_config": {"mode": "closed", "agents": ["qwen3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_closed", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 605, "message_tokens": 509, "message_utf8_bytes": 1189, "cache_tensor_bytes": null, "serialized_payload_bytes": 1189, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 14417.945208027959, "hops": {"1": 7388.310500071384, "2": 7029.634707956575}, "receiver_prefill_ms": 74.4137919973582, "ttft_ms": 13.573125004768372, "decode_ms": 13.354499940760434}, "total_wall_ms": 14519.808249082416, "ttft_ms": 13.573125004768372, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1196}, "2": {"facts_total": 64, "facts_retained": 28, "retention_rate": 0.4375, "retention_by_kind": {"person": 0.6666666666666666, "date": 0.0, "amount": 0.42857142857142855, "status": 0.75, "place": 0.0, "identifier": 0.5, "quantity": 0.5}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1189}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "49b54c3c28297dd6f3503e10cc594cea5454825636b39e95a6e8663b6c37ed71", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
31 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "ef1cb0065f7c97faa4ef549b", "attempt_id": "4897ddef5806", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.381200Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 426, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 72.67737505026162, "ttft_ms": 13.352832989767194, "decode_ms": 13.35733407177031}, "total_wall_ms": 7289.022208889946, "ttft_ms": 13.352832989767194, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
32 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "5aead85d240368a08066c2c3", "attempt_id": "be42e414d32b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.500359Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 432, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 124.0327499108389, "ttft_ms": 14.377749990671873, "decode_ms": 13.802209054119885}, "total_wall_ms": 7341.845458839089, "ttft_ms": 14.377749990671873, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "55c1fbf7e46378e1c4bdf6ad", "attempt_id": "3bdbc43d86b9", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.670566Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 432, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.62875002902001, "ttft_ms": 13.563584070652723, "decode_ms": 13.158750021830201}, "total_wall_ms": 7269.9129588436335, "ttft_ms": 13.563584070652723, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
34 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "15bba002b57c9391083f8b1a", "attempt_id": "134a07c6c081", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.770854Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 429, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.7230420159176, "ttft_ms": 13.315167045220733, "decode_ms": 13.420167029835284}, "total_wall_ms": 7269.994708942249, "ttft_ms": 13.315167045220733, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
35 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "75dc721488e0ef1126081271", "attempt_id": "bed9a2a1f704", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:28.869037Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 437, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 74.69737506471574, "ttft_ms": 13.112541986629367, "decode_ms": 144.9349169852212}, "total_wall_ms": 7422.3342089680955, "ttft_ms": 13.112541986629367, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 11}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172 NOK", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "ce5dae2b035af8d38bda9e7d1f1cb7eb7e4141e20298d86d8bce5fe5ac6c0246"}
36 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "3e9a22c3f5104a1eddd41703", "attempt_id": "78399ab6c557", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.118864Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 435, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 64.27808292210102, "ttft_ms": 13.261125073768198, "decode_ms": 132.68429203890264}, "total_wall_ms": 7399.776375852525, "ttft_ms": 13.261125073768198, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "BH-71Y-2851", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "38b55d8642902ac72f5987064e67fbe9b766a04dc338df11f6fa45d4cb64179e"}
37 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "eb3ecdabac7df3aa56f2db7d", "attempt_id": "99f7f5ca3819", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.346863Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 429, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.109832922928035, "ttft_ms": 13.361750054173172, "decode_ms": 13.09916703030467}, "total_wall_ms": 7269.102249876596, "ttft_ms": 13.361750054173172, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "no", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "9390298f3fb0c5b160498935d79cb139aef28e1c47358b4bbba61862b9c26e59"}
38 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "1adfc40d070513eb8dcaafa3", "attempt_id": "c00fc0794f37", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.443663Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 427, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 67.15850008185953, "ttft_ms": 13.201624969951808, "decode_ms": 13.237791019491851}, "total_wall_ms": 7283.145666937344, "ttft_ms": 13.201624969951808, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
39 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "119e16c4305375bb8dc1f5c3", "attempt_id": "047930cf5767", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.554627Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 443, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 112.22383310087025, "ttft_ms": 13.389540952630341, "decode_ms": 13.051833026111126}, "total_wall_ms": 7328.250583843328, "ttft_ms": 13.389540952630341, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
40 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "979bd81c65b6db191a8e985d", "attempt_id": "c54b44cc375b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.711472Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 435, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.7681250134483, "ttft_ms": 13.247541966848075, "decode_ms": 13.039291952736676}, "total_wall_ms": 7269.584166933782, "ttft_ms": 13.247541966848075, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
41 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "065a7e3fa74c1ad482af1d3e", "attempt_id": "8fd72b5c73ee", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.809877Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 444, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 110.29041605070233, "ttft_ms": 13.615458039566875, "decode_ms": 13.41954106464982}, "total_wall_ms": 7326.904041925445, "ttft_ms": 13.615458039566875, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
42 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "91f86e20ea07bbdb391d30a6", "attempt_id": "fff9ed3ee7b5", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:29.964638Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 439, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.64233395084739, "ttft_ms": 13.906834064982831, "decode_ms": 94.21266708523035}, "total_wall_ms": 7351.373499841429, "ttft_ms": 13.906834064982831, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1418934", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b42460a96a2ffc3ec3bc3e8bb110e80b04f38ffa2f4006e5a3159cebdee13b7f"}
43 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "4c81006b8214f8778abfbac8", "attempt_id": "c28310477a2b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:30.144231Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 434, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 68.86416592169553, "ttft_ms": 13.305291067808867, "decode_ms": 13.282250030897558}, "total_wall_ms": 7285.093041951768, "ttft_ms": 13.305291067808867, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
44 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "30262d87fea47fb55f4c9228", "attempt_id": "eea79bc96c46", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:30.258590Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 439, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.78554202616215, "ttft_ms": 13.282875064760447, "decode_ms": 13.299459009431303}, "total_wall_ms": 7269.934750860557, "ttft_ms": 13.282875064760447, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
45 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e2fa4da7d7595a735512203b", "attempt_id": "4e5fc7bd7c56", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h1", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:30.357961Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 1, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 440, "message_tokens": 344, "message_utf8_bytes": 817, "cache_tensor_bytes": null, "serialized_payload_bytes": 817, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 7189.094333909452, "hops": {"1": 7189.094333909452}, "receiver_prefill_ms": 53.51837503258139, "ttft_ms": 13.22641596198082, "decode_ms": 13.591583934612572}, "total_wall_ms": 7269.903499865904, "ttft_ms": 13.22641596198082, "input_usage": {"lookups": 0}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "448fbefbbf3dc2a8c9785c435f1c717947c9d692820d830c64b908b542a8566e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
46 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "8888774dc820faf67464763e", "attempt_id": "bd23292acf1f", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.360272Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q00_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 587, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 77.82762497663498, "ttft_ms": 13.771916972473264, "decode_ms": 13.905708095990121}, "total_wall_ms": 15194.374041981064, "ttft_ms": 13.771916972473264, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
47 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "7f668e1f7b3eb3d9b9fd3057", "attempt_id": "e595c673de30", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.483820Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q01_single_fact", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 592, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 74.5352499652654, "ttft_ms": 13.96233297418803, "decode_ms": 13.73537490144372}, "total_wall_ms": 15190.9696678631, "ttft_ms": 13.96233297418803, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
48 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "070822fa247dd66d78b6ffcd", "attempt_id": "6d31e2b2d76b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.603556Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q02_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 592, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 73.35308298934251, "ttft_ms": 14.057875028811395, "decode_ms": 13.937833020463586}, "total_wall_ms": 15190.103625995107, "ttft_ms": 14.057875028811395, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
49 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "1c3f890e0a19b625c9227879", "attempt_id": "539634270a2e", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.721629Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q03_exact_copy", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 589, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 74.27520793862641, "ttft_ms": 14.271625084802508, "decode_ms": 13.794749975204468}, "total_wall_ms": 15191.149875987321, "ttft_ms": 14.271625084802508, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
50 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "5c48647a9e1b348b9b640926", "attempt_id": "50e5bb1c3127", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:38.840822Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 598, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 77.12575001642108, "ttft_ms": 14.055041945539415, "decode_ms": 69.60441707633436}, "total_wall_ms": 15249.57975000143, "ttft_ms": 14.055041945539415, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
51 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "e6a0f8ca62392e868ec815b1", "attempt_id": "80886cabcadb", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.019147Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 595, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 82.5685829622671, "ttft_ms": 14.223375008441508, "decode_ms": 14.372208039276302}, "total_wall_ms": 15199.900208972394, "ttft_ms": 14.223375008441508, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
52 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "05fe5b3197f86364deed50a4", "attempt_id": "413eadf2083b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.148069Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q06_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 589, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 84.24354100134224, "ttft_ms": 14.065457973629236, "decode_ms": 13.865416985936463}, "total_wall_ms": 15200.890749925748, "ttft_ms": 14.065457973629236, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "No", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "1ea442a134b2a184bd5d40104401f2a37fbc09ccf3f4bc9da161c6099be3691d"}
53 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "069d1a413855b163acd583f6", "attempt_id": "5a23a496b605", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.276600Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q07_negation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 587, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 86.33862505666912, "ttft_ms": 14.2137500224635, "decode_ms": 13.89129098970443}, "total_wall_ms": 15203.177750925533, "ttft_ms": 14.2137500224635, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "No", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "1ea442a134b2a184bd5d40104401f2a37fbc09ccf3f4bc9da161c6099be3691d"}
54 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "caeca776a64b5957737b1f15", "attempt_id": "1d487926748b", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.407745Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 603, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 91.23012493364513, "ttft_ms": 14.472333015874028, "decode_ms": 13.88770795892924}, "total_wall_ms": 15208.31024996005, "ttft_ms": 14.472333015874028, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
55 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "db28675b1125cfdf5b610b3d", "attempt_id": "0765c60f69f5", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.543531Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 595, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 93.1548330700025, "ttft_ms": 14.234708971343935, "decode_ms": 14.128541923128068}, "total_wall_ms": 15210.25970892515, "ttft_ms": 14.234708971343935, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
56 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "06f0097c3db73ac91f6d5b1f", "attempt_id": "dcb96516ced4", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.681362Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 604, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 90.80570796504617, "ttft_ms": 14.129042043350637, "decode_ms": 14.112583012320101}, "total_wall_ms": 15207.776958937757, "ttft_ms": 14.129042043350637, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
57 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "ba7d480a807e039f9edb854f", "attempt_id": "b01c7d2c7f31", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:39.816641Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 600, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 89.42750003188848, "ttft_ms": 14.108083094470203, "decode_ms": 68.82829195819795}, "total_wall_ms": 15261.129499878734, "ttft_ms": 14.108083094470203, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7,280,585", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "2b81d1bd559c98218f68f4bc6bf62ababe33d07bcf6ec2cd6c2cd0508c0e52b5"}
58 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "b051902bc46dc13e0a1bf186", "attempt_id": "c6ae50cac079", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:40.005425Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q12_unit_conversion", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 595, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 87.34066702891141, "ttft_ms": 14.235583948902786, "decode_ms": 13.789458898827434}, "total_wall_ms": 15204.090374871157, "ttft_ms": 14.235583948902786, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
59 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "d011ea48a6559b63fe4420d2", "attempt_id": "183049fa628d", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:40.137750Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 600, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 84.58429202437401, "ttft_ms": 14.115125057287514, "decode_ms": 14.149583061225712}, "total_wall_ms": 15201.539375935681, "ttft_ms": 14.115125057287514, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
60 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E4_smoke", "trial_id": "46883052d3de12af358f9963", "attempt_id": "cbe9b3f834c6", "run_group_id": "E4_smoke:A5|repairable|qwen3b-llama3b:h2", "protocol_hash": "unfrozen", "config_hash": "6b824f39d528ca63caa404b46d54b529ed8a1bc08e5cc1584cda4d73da84aea9", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:40.266732Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11|mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "receiver_model_revision": "mlx-community/Llama-3.2-3B-Instruct-bf16@6d88ba43024fef71b10e52e101c7cd4598322601", "tokenizer_hashes": {"receiver": "b9d8872cf3a99da99e1fc4b0f71232a451632dc49ca4c6771549b4efa5c1c763"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "A5", "implementation_fidelity": "custom", "compression_config": {"mode": "repairable", "agents": ["qwen3b", "llama3b"], "budget_tokens": 512}, "question_access": "blind", "topology": "chain_repairable", "handoff_index": 2, "reuse_count": 15, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1766, "query_tokens": 601, "message_tokens": 484, "message_utf8_bytes": 1403, "cache_tensor_bytes": null, "serialized_payload_bytes": 1403, "side_information_bytes": 0, "fetched_bytes": 540, "transport_bytes_if_measured": null, "timing_spans": {"chain_ms": 15088.201458915137, "hops": {"1": 7189.094333909452, "2": 7899.107125005685}, "receiver_prefill_ms": 86.56408300157636, "ttft_ms": 13.959750067442656, "decode_ms": 13.88420898001641}, "total_wall_ms": 15203.10266688466, "ttft_ms": 13.959750067442656, "input_usage": {"lookups": 1}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"ledger": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}, "ledger_by_hop": {"1": {"facts_total": 64, "facts_retained": 10, "retention_rate": 0.15625, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.125, "amount": 0.14285714285714285, "status": 0.3333333333333333, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 817}, "2": {"facts_total": 64, "facts_retained": 16, "retention_rate": 0.25, "retention_by_kind": {"person": 0.16666666666666666, "date": 0.25, "amount": 0.14285714285714285, "status": 0.75, "place": 0.0, "identifier": 0.08333333333333333, "quantity": 0.16666666666666666}, "unsupported_numbers": 0, "unsupported_identifiers": 0, "distorted_sentences": 0, "message_bytes": 1403}}}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E4_smoke.log", "payload_hash": "1cc02e5fe068336092e7eb9c927276603bff34770095beec1e4af3e6ef63a8cd", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
added runs/raw/E5_smoke.jsonl +80 −0
@@ -0,0 +1,80 @@
1 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "c2531f85c95dbc57153d0f4e", "attempt_id": "6ee0f1aa0a77", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:53.622662Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2417, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 15.056415926665068, "decode_ms": 121.93570903036743, "receiver_prefill_ms": 287.17620798852295}, "total_wall_ms": 434.8010838730261, "ttft_ms": 15.056415926665068, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
2 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "1ed0773614be3aba3c8c6a13", "attempt_id": "58a507aa13dc", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:54.074634Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2415, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.456250075250864, "decode_ms": 134.57829202525318, "receiver_prefill_ms": 263.9893329469487}, "total_wall_ms": 415.5645000282675, "ttft_ms": 13.456250075250864, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
3 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0dd784899f055a526e5323fe", "attempt_id": "e4485ad562f9", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:54.505023Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2423, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.834625016897917, "decode_ms": 95.39945796132088, "receiver_prefill_ms": 272.29566604364663}, "total_wall_ms": 384.97762486804277, "ttft_ms": 13.834625016897917, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "7767550", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "87710bb4bb8c5f86714a007462705923c5ce8009326882c4a0f838a7b0005f5c"}
4 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e464b617bf45386c12948550", "attempt_id": "2c77b344b5b6", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:54.904815Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2415, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.825792004354298, "decode_ms": 122.0632919576019, "receiver_prefill_ms": 304.7115419758484}, "total_wall_ms": 444.1995839588344, "ttft_ms": 13.825792004354298, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1,384,522", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "086bd56bc8b1d63bbe425b997da59a545e1bd7ffa6bdd732e726b19220cf2875"}
5 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0f5261a73db9951a26f35b2a", "attempt_id": "967381c85889", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:55.364219Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2424, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 14.14341595955193, "decode_ms": 68.84212500881404, "receiver_prefill_ms": 304.8282921081409}, "total_wall_ms": 391.55841700267047, "ttft_ms": 14.14341595955193, "input_usage": {}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Zenmel-18", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "d324766044a69218e7c53a2be92c812e7edad5aa0821701308fad4d0b3f63e2d"}
6 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "35196571a68a7930a7576c47", "attempt_id": "80b23bba763b", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:55.770805Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2419, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.75274988822639, "decode_ms": 135.45058306772262, "receiver_prefill_ms": 310.9101669397205}, "total_wall_ms": 463.7714169220999, "ttft_ms": 13.75274988822639, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "14,243,807", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "9889c2bdd5bbe834773eb956d4693d3a4e8109c870df394ced69710b96da2b49"}
7 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "84ccc3d49d6380ed8160628a", "attempt_id": "809163a36d9e", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:56.250012Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2419, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.833459001034498, "decode_ms": 13.668250059708953, "receiver_prefill_ms": 295.51949992310256}, "total_wall_ms": 326.49495790246874, "ttft_ms": 13.833459001034498, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
8 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6e8f0133c433a07dfd1271b6", "attempt_id": "ce174c6ef5a7", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:34:56.591948Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 2420, "message_tokens": 2329, "message_utf8_bytes": 8318, "cache_tensor_bytes": null, "serialized_payload_bytes": 8318, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.004541943781077862, "ttft_ms": 13.64783407188952, "decode_ms": 13.888042070902884, "receiver_prefill_ms": 317.6879999227822}, "total_wall_ms": 348.7097079632804, "ttft_ms": 13.64783407188952, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "2eb956b77c113d21ef86e1ff1180f246b6529da5ba1a27c8716138d5e49803d1", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
9 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6f547971ac924941a7009148", "attempt_id": "2572439e4fb2", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.103232Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1127, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 14.00866697076708, "decode_ms": 93.82133395411074, "receiver_prefill_ms": 121.81266699917614}, "total_wall_ms": 14379.160082899034, "ttft_ms": 14.00866697076708, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6889172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "adcb8d196080006451756ad2fe2106d5a1b6500c0f76ffaa05d7fe704243ad3b"}
10 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e403e9aca77bdb244fd0b4d0", "attempt_id": "d1c277a13a64", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.351018Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1125, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.991833897307515, "decode_ms": 135.3418749058619, "receiver_prefill_ms": 123.2716670492664}, "total_wall_ms": 14421.900999965146, "ttft_ms": 13.991833897307515, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
11 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6baa6b01063b68303a554b17", "attempt_id": "53d6a69496ae", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.641096Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1133, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.811749988235533, "decode_ms": 93.77762500662357, "receiver_prefill_ms": 123.21070895995945}, "total_wall_ms": 14380.13870792929, "ttft_ms": 13.811749988235533, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6068030", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "ce3d061151c7d10d9cc6f1a389ccecd8961b5760dff6c384dfc7a02e8f9222bf"}
12 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "4dd6c587acad2808d5612467", "attempt_id": "93b5e576057f", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:11.889706Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1125, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.429375016130507, "decode_ms": 120.65325002186, "receiver_prefill_ms": 121.36654101777822}, "total_wall_ms": 14404.811999876983, "ttft_ms": 13.429375016130507, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1,355,485", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "064c9918faa4dc34ddf1b7fd309342033e0c55a27757697cfac056328d0c209e"}
13 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "83d1817abd7e526590a607fc", "attempt_id": "722e64bac111", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.163005Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1134, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.22362502105534, "decode_ms": 67.09000002592802, "receiver_prefill_ms": 121.55058397911489}, "total_wall_ms": 14351.148915942758, "ttft_ms": 13.22362502105534, "input_usage": {}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Zenmel-18", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "d324766044a69218e7c53a2be92c812e7edad5aa0821701308fad4d0b3f63e2d"}
14 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "110a9f2ab259f054abb78608", "attempt_id": "743526c9852a", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.382194Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1129, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.553333934396505, "decode_ms": 134.5761250704527, "receiver_prefill_ms": 120.83662499208003}, "total_wall_ms": 14418.322874000296, "ttft_ms": 13.553333934396505, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "14,203,472", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "1b8d63c935e4628d5567478aa9f1aa1d898d43d55f5be209ba0f208bd190f37a"}
15 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0f99ec00589286ae4ab96da3", "attempt_id": "509282236e10", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.668873Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1129, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.50362494122237, "decode_ms": 13.378208968788385, "receiver_prefill_ms": 121.44350004382432}, "total_wall_ms": 14297.648957930505, "ttft_ms": 13.50362494122237, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
16 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "74f4b20f100fb255543e2545", "attempt_id": "8e0fb07da355", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:12.834845Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1130, "message_tokens": 1037, "message_utf8_bytes": 2536, "cache_tensor_bytes": null, "serialized_payload_bytes": 2536, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14147.489207913168, "ttft_ms": 13.468041084706783, "decode_ms": 13.543165987357497, "receiver_prefill_ms": 121.2417499627918}, "total_wall_ms": 14297.607791842893, "ttft_ms": 13.468041084706783, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "76cdd4092824810f26e896fecca8b49011db2d64024b0c874dc8ab69cc7bb1b9", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
17 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ae6de40e70115e38ec1fe95f", "attempt_id": "b27acf424638", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.166118Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 694, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 17.047167057171464, "decode_ms": 120.8459580084309, "receiver_prefill_ms": 77.17541698366404}, "total_wall_ms": 14381.83108298108, "ttft_ms": 17.047167057171464, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6,889,172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "76875b24cee85c9daa41a3b2716f4617e3b85d1aa6a8a5fd34f349dcf4ae3d7e"}
18 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "db1e0004d9835d1cc531df6e", "attempt_id": "3d8a1e931f8d", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.398391Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 692, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.37675005197525, "decode_ms": 133.19487508852035, "receiver_prefill_ms": 76.64595905225724}, "total_wall_ms": 14389.950957032852, "ttft_ms": 13.37675005197525, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
19 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "9a09901f690e45b81ec42ccb", "attempt_id": "279dae943ccf", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.638797Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 700, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 14.127249945886433, "decode_ms": 14.26125003490597, "receiver_prefill_ms": 77.59662496391684}, "total_wall_ms": 14272.744624060579, "ttft_ms": 14.127249945886433, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
20 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "f4cc5af90e025a3438c9375e", "attempt_id": "9f2ae68b8d64", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.761926Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 692, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.230124954134226, "decode_ms": 13.14816705416888, "receiver_prefill_ms": 76.50433294475079}, "total_wall_ms": 14269.677623989992, "ttft_ms": 13.230124954134226, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
21 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "72f5efac4d38ab5dc0654048", "attempt_id": "b72da5e5172f", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:27.884509Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 701, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.496833038516343, "decode_ms": 66.74774992279708, "receiver_prefill_ms": 76.85845799278468}, "total_wall_ms": 14323.815457057208, "ttft_ms": 13.496833038516343, "input_usage": {}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
22 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "550b173d9301c38240466802", "attempt_id": "94c23122ea16", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:28.058990Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 696, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.47495790105313, "decode_ms": 93.26966700609773, "receiver_prefill_ms": 76.70991704799235}, "total_wall_ms": 14350.178999011405, "ttft_ms": 13.47495790105313, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1415087", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "d8de98946f48b78af60397a05b6e577a8e2416856d227f24711ff800542ffa6a"}
23 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7d1e524827c3ee1a292be484", "attempt_id": "8aa8ea1aba50", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:28.259720Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 696, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.402125099673867, "decode_ms": 13.456832966767251, "receiver_prefill_ms": 76.26687502488494}, "total_wall_ms": 14269.774833112024, "ttft_ms": 13.402125099673867, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
24 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "f35cff3ff329915847b85b7f", "attempt_id": "9af080cdeacc", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:28.380203Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 15}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 697, "message_tokens": 601, "message_utf8_bytes": 1647, "cache_tensor_bytes": null, "serialized_payload_bytes": 1647, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14165.317791048437, "ttft_ms": 13.48324993159622, "decode_ms": 13.602624996565282, "receiver_prefill_ms": 78.33091600332409}, "total_wall_ms": 14272.0727911219, "ttft_ms": 13.48324993159622, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ada8b0bc50cda511754ddf71b55cdca0747aa6cfc3ca64acd417bb55aa528bc6", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
25 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "8843c4e0a2c64e04f79d6aa2", "attempt_id": "1496e16b85c7", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:42.643857Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1384, "message_tokens": 1292, "message_utf8_bytes": 3380, "cache_tensor_bytes": null, "serialized_payload_bytes": 3380, "side_information_bytes": 0, "fetched_bytes": 1683, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.578542042523623, "decode_ms": 94.64470902457833, "receiver_prefill_ms": 148.28600001055747}, "total_wall_ms": 14400.122708058916, "ttft_ms": 13.578542042523623, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6889172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "4d600cf3416bdfdbcceef02eedaf880d5773ede73b4ba1dca054e616d4012ee4", "output_hash": "adcb8d196080006451756ad2fe2106d5a1b6500c0f76ffaa05d7fe704243ad3b"}
26 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0f2933272401cc084370967a", "attempt_id": "0e2fc0d8f610", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:42.918501Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1354, "message_tokens": 1264, "message_utf8_bytes": 3303, "cache_tensor_bytes": null, "serialized_payload_bytes": 3303, "side_information_bytes": 0, "fetched_bytes": 1613, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 14.11025004927069, "decode_ms": 134.67420800589025, "receiver_prefill_ms": 144.1560419043526}, "total_wall_ms": 14436.438375036232, "ttft_ms": 14.11025004927069, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "SP-38G-4217", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "b84ec3106dafb26defcbe7f76d40bd644d33cd7223b6d910a9985a77d2e932b5", "output_hash": "e643c21a78559b96fd65325d9ce779b7b8fd6a47c53cf7c2cda1efa8412458c4"}
27 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "89db5fdff414eca1360c5c8b", "attempt_id": "eaef034bb8fe", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.229874Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1415, "message_tokens": 1317, "message_utf8_bytes": 3428, "cache_tensor_bytes": null, "serialized_payload_bytes": 3428, "side_information_bytes": 0, "fetched_bytes": 1727, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.573833974078298, "decode_ms": 13.344207894988358, "receiver_prefill_ms": 152.19508297741413}, "total_wall_ms": 14322.535209008493, "ttft_ms": 13.573833974078298, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "895593c9e120f4063d074d72e0ea36606dc5b4bcacefea4ff70ab64562695bed", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
28 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "cb80418840e30b25cabc7ece", "attempt_id": "7f0458b9139d", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.426967Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1407, "message_tokens": 1317, "message_utf8_bytes": 3428, "cache_tensor_bytes": null, "serialized_payload_bytes": 3428, "side_information_bytes": 0, "fetched_bytes": 1727, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.562500011175871, "decode_ms": 13.312124996446073, "receiver_prefill_ms": 149.3196670198813}, "total_wall_ms": 14319.655666942708, "ttft_ms": 13.562500011175871, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "895593c9e120f4063d074d72e0ea36606dc5b4bcacefea4ff70ab64562695bed", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
29 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7c8760d267881299e3e69f39", "attempt_id": "55271e101d9f", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.620845Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1374, "message_tokens": 1275, "message_utf8_bytes": 3299, "cache_tensor_bytes": null, "serialized_payload_bytes": 3299, "side_information_bytes": 0, "fetched_bytes": 1615, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.59579199925065, "decode_ms": 67.70854198839515, "receiver_prefill_ms": 145.4034170601517}, "total_wall_ms": 14370.145458029583, "ttft_ms": 13.59579199925065, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 5}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Torsel-5", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "a6f42a821d7943c352386b8b4e20945501dc7944212ceca382a93e86f0088dfd", "output_hash": "0a2de4a55396e5f7b4d24e988f7452e33fe7e43fd6e05881b80bcb3034d74d37"}
30 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "094399c234fd0de4c00855a5", "attempt_id": "bd15bd1e7d12", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:43.865663Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1411, "message_tokens": 1317, "message_utf8_bytes": 3428, "cache_tensor_bytes": null, "serialized_payload_bytes": 3428, "side_information_bytes": 0, "fetched_bytes": 1727, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.63187504466623, "decode_ms": 134.21929103787988, "receiver_prefill_ms": 151.07270807493478}, "total_wall_ms": 14442.394666024484, "ttft_ms": 13.63187504466623, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "13,751,547", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "7f0d0a79d177c4ebee46fa47933d0ad0511ad6f52d414fd7a3f00743886aa803", "output_hash": "8381b7b8800f64937ec3e0e0d15cdbaf1f2a76bc929b403929b108f2390ad082"}
31 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "40405206bd7a456c61f33574", "attempt_id": "c55532797cd4", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:44.182787Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1378, "message_tokens": 1284, "message_utf8_bytes": 3373, "cache_tensor_bytes": null, "serialized_payload_bytes": 3373, "side_information_bytes": 0, "fetched_bytes": 1673, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.48029205109924, "decode_ms": 13.330582994967699, "receiver_prefill_ms": 148.3127500396222}, "total_wall_ms": 14318.527041003108, "ttft_ms": 13.48029205109924, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "1edfcca5d5af0aeecdf3e6cfce6ea87f864faa495a0061a6b08bd063c3b5e9fa", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
32 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "1eeba3fb9eed49bfded9ff9b", "attempt_id": "c6a2e2770983", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:44.375877Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 1392, "message_tokens": 1297, "message_utf8_bytes": 3392, "cache_tensor_bytes": null, "serialized_payload_bytes": 3392, "side_information_bytes": 0, "fetched_bytes": 1691, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14141.30474999547, "ttft_ms": 13.490041019394994, "decode_ms": 13.273874996230006, "receiver_prefill_ms": 149.1100420244038}, "total_wall_ms": 14319.356792024337, "ttft_ms": 13.490041019394994, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "cdee7b9b411dca0cbee7e59f64760f191ceadb11216c0c15f6760e3d5d53486e", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
33 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "d8bc85caf71ca9f56f3f8d74", "attempt_id": "cd7dca1dd9f6", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:44.842542Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 44, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 63.81474994122982, "decode_ms": 139.43170895799994, "receiver_prefill_ms": 0.0}, "total_wall_ms": 481.60183301661164, "ttft_ms": 63.81474994122982, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n6889172", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b9fe7ad55b0c37fb0f5551b72bd99f8cf6f925fd75264f28755ecac8f7652a24"}
34 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "cc10dfa190405328babb888b", "attempt_id": "5b11e161d16d", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.067430Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 42, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.505166061222553, "decode_ms": 182.42304190061986, "receiver_prefill_ms": 0.0}, "total_wall_ms": 485.9277909854427, "ttft_ms": 28.505166061222553, "input_usage": {}, "output_usage": {"output_tokens": 13}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nBH-71Y-2851", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "5551fa62e27e66d44004c639f2de126fbbaadc134a344e5893b9f159865d21a1"}
35 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6bc7337f68d872dfed13b3aa", "attempt_id": "445efebe981c", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.297490Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 50, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.82962499279529, "decode_ms": 140.16608300153166, "receiver_prefill_ms": 0.0}, "total_wall_ms": 443.9334579510614, "ttft_ms": 28.82962499279529, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n7767550", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "3d29dc8364bc9f25a62fdf8895b2508164bb30eff468c6f312e1acc164bfae1d"}
36 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0e411de44dbcd4f72847a3c0", "attempt_id": "0e4e9f4e32b8", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.484772Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 42, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.589040972292423, "decode_ms": 139.41237493418157, "receiver_prefill_ms": 0.0}, "total_wall_ms": 442.85183295141906, "ttft_ms": 28.589040972292423, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n7354655", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "e3a1a67b5dd99a3b78bca513a9fdf83915d81557a4b4dd49abdc6922943cfdb4"}
37 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0b0b601028268214da9df263", "attempt_id": "49dab304374f", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.671251Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 51, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.831708943471313, "decode_ms": 114.26591593772173, "receiver_prefill_ms": 0.0}, "total_wall_ms": 417.0399159193039, "ttft_ms": 28.831708943471313, "input_usage": {}, "output_usage": {"output_tokens": 8}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nTorsel-5", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "98076f21caa823e0185fe09260512bc5a273f6343b0926f76c11ad1fe07fce1e"}
38 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ede55d225cc5358e7f48175d", "attempt_id": "236168bc7697", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:45.831997Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 46, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.3097920473665, "decode_ms": 181.39916693326086, "receiver_prefill_ms": 0.0}, "total_wall_ms": 484.59966597147286, "ttft_ms": 28.3097920473665, "input_usage": {}, "output_usage": {"output_tokens": 13}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n12,203,327", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "0bfff1a709c4525e7b99586414c4d2d619ea005a7726c0fff6cfea88ba70e322"}
39 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7510675c1038c0e10681915c", "attempt_id": "fa8fcd0ab2bc", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.060697Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 46, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.092040913179517, "decode_ms": 13.7310839490965, "receiver_prefill_ms": 0.0}, "total_wall_ms": 316.7332489974797, "ttft_ms": 28.092040913179517, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
40 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "048dde7c876767f1c7b6ed64", "attempt_id": "101a4e0c2969", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.120774Z", "task_family": "synthetic", "document_id": "syn_02000_2000", "document_hash": "3bcf1bb02792129efca162575bc2ef64622b41083d4e2870fed106d663e6417c", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02000_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88731648}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1971, "query_tokens": 47, "message_tokens": null, "message_utf8_bytes": 8292, "cache_tensor_bytes": 88731648, "serialized_payload_bytes": 88731648, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 272.5884160026908, "ttft_ms": 28.440083959139884, "decode_ms": 13.840583036653697, "receiver_prefill_ms": 0.0}, "total_wall_ms": 317.2282080631703, "ttft_ms": 28.440083959139884, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Zennovvol-6", "bogus_amount": "6,770,940", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02000_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
41 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "77c6e32c934f47a56eac7a42", "attempt_id": "689ef527ca9d", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.258781Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2410, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 14.075792045332491, "decode_ms": 96.16091707721353, "receiver_prefill_ms": 265.7509580021724}, "total_wall_ms": 379.7228749608621, "ttft_ms": 14.075792045332491, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1844550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "23de9db5e2c3413d7142a9e77e743ed168ec6de65abf6413935059d107bb8c17"}
42 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "db3b581ff79d7f09f6bf4ab9", "attempt_id": "5b153f170fbe", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:46.653865Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2413, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.834582990966737, "decode_ms": 137.00370804872364, "receiver_prefill_ms": 265.40816691704094}, "total_wall_ms": 419.9756659800187, "ttft_ms": 13.834582990966737, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "YA-41S-3303", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "2ae4249ea3f767dbc2fc15d5f075c3c9c1ff92b50a630df9d3adb6416607262b"}
43 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ab3ed3250eda6c012c9f752e", "attempt_id": "ecfff760d068", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:47.089921Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2418, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.557915925048292, "decode_ms": 122.66433401964605, "receiver_prefill_ms": 265.6623750226572}, "total_wall_ms": 405.5574160302058, "ttft_ms": 13.557915925048292, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "2,448,885", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "d9cc7ba7227735fd33d7411dda32576c50b6778cb7717cddc1c13d9b861dcc49"}
44 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "6f4387c55cf8a5e9242fe62d", "attempt_id": "848e0a7fab23", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:47.511113Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2410, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.478250009939075, "decode_ms": 122.04145907890052, "receiver_prefill_ms": 263.73116695322096}, "total_wall_ms": 402.9229579027742, "ttft_ms": 13.478250009939075, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "2,968,560", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "e16428fe643f862e47e9569d5ec0df386c633fbbeb1a5fdf347a38816d01f1bf"}
45 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "ecd0325ac0925265de852893", "attempt_id": "58abafd92f85", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:47.929034Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2420, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.6748329969123, "decode_ms": 80.88966691866517, "receiver_prefill_ms": 264.1993750585243}, "total_wall_ms": 362.57595801725984, "ttft_ms": 13.6748329969123, "input_usage": {}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "d2ace64e89bac774483fbdf2c3b630bf9c232f8f7b2aa06eda5f4a26d74f7cf0"}
46 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "f18f4cfed64113435b36d3ec", "attempt_id": "43eb6195f28e", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:48.307303Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2414, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.73612496536225, "decode_ms": 95.92591703403741, "receiver_prefill_ms": 263.4170410456136}, "total_wall_ms": 376.7490410245955, "ttft_ms": 13.73612496536225, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6447515", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "854fcfa0ae827910332f74fc824602d36fc688e4c310ef595c93498a3e50d16c"}
47 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "adc3d4c18d3e1946ad090c8e", "attempt_id": "f66de66b649f", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:48.699352Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2414, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 14.189749956130981, "decode_ms": 13.982250005938113, "receiver_prefill_ms": 263.9900000067428}, "total_wall_ms": 295.8700000308454, "ttft_ms": 14.189749956130981, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
48 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "bae716a32c678e30f1c7ba76", "attempt_id": "78c87b873f62", "run_group_id": "E5_smoke:full_context|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:35:49.010711Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "full_context", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 2416, "message_tokens": 2325, "message_utf8_bytes": 8308, "cache_tensor_bytes": null, "serialized_payload_bytes": 8308, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 0.026708003133535385, "ttft_ms": 13.75416701193899, "decode_ms": 13.620915939100087, "receiver_prefill_ms": 264.1928329830989}, "total_wall_ms": 295.36325007211417, "ttft_ms": 13.75416701193899, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "0b3b8e65ab8a14af9771a387a6e3c9f0b249b61b00b4f444f0fef62d2299d0aa", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
49 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e7141d455cd74ac0f9aaaf24", "attempt_id": "be4d577aba0b", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:03.456810Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1124, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.593790936283767, "decode_ms": 122.23579199053347, "receiver_prefill_ms": 122.03633296303451}, "total_wall_ms": 14394.545415998437, "ttft_ms": 13.593790936283767, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1,844,550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "c3a824f75c26a69ab7e2a0dfde5b194c586a879a105eff635ab136cdb1a4fd6c"}
50 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a32aeec79a9ee8aa0432446f", "attempt_id": "fc67be95d1d7", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:03.732536Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1127, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.491875026375055, "decode_ms": 133.8763750391081, "receiver_prefill_ms": 121.43004103563726}, "total_wall_ms": 14405.404999968596, "ttft_ms": 13.491875026375055, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "PN-55H-5346", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "07e77c9600b8c4977df13983dd328c2c2a77178cdbd678cc76a9cc9784f43993"}
51 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "2c3aab7cdf9d14370dc39bab", "attempt_id": "56d1872e03bc", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.019008Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1132, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.250541989691556, "decode_ms": 93.89745793305337, "receiver_prefill_ms": 121.74612504895777}, "total_wall_ms": 14365.498958039097, "ttft_ms": 13.250541989691556, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3013170", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "a44e28276444065cad5bf760a512ece505a4d855205b9b2bb30a4479693b0a9f"}
52 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7feba5fa129f9153dfaef9f2", "attempt_id": "d6e23cc74256", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.265580Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1124, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.742499984800816, "decode_ms": 122.71799996960908, "receiver_prefill_ms": 121.24983302783221}, "total_wall_ms": 14394.36841593124, "ttft_ms": 13.742499984800816, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "2,968,560", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "e16428fe643f862e47e9569d5ec0df386c633fbbeb1a5fdf347a38816d01f1bf"}
53 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a3d11c9a04eef04742acef19", "attempt_id": "f15db681d709", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.541338Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1134, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.612332986667752, "decode_ms": 82.18583289999515, "receiver_prefill_ms": 123.20812500547618}, "total_wall_ms": 14355.694625061005, "ttft_ms": 13.612332986667752, "input_usage": {}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "d2ace64e89bac774483fbdf2c3b630bf9c232f8f7b2aa06eda5f4a26d74f7cf0"}
54 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "acd92f874c5b04045fba8673", "attempt_id": "57634f0c7ac6", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:04.777605Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1128, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.54733295738697, "decode_ms": 94.51099997386336, "receiver_prefill_ms": 130.06750005297363}, "total_wall_ms": 14374.922958086245, "ttft_ms": 13.54733295738697, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1449880", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "60b3b6a51e6e9c252943d500d091109937474d1e7422315c117bcba390d4bd51"}
55 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "80c9b8be2a765c02de01ed2b", "attempt_id": "2ff500b209fc", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:05.034415Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1128, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.84162507019937, "decode_ms": 13.439041911624372, "receiver_prefill_ms": 134.0613749343902}, "total_wall_ms": 14298.041917034425, "ttft_ms": 13.84162507019937, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
56 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "024b670d45a38870b1612c72", "attempt_id": "49bcc2b78e8c", "run_group_id": "E5_smoke:concat_A4|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:05.213930Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "concat_A4", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "sender_tokens": 1024}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1130, "message_tokens": 1037, "message_utf8_bytes": 2967, "cache_tensor_bytes": null, "serialized_payload_bytes": 2967, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14134.683375013992, "ttft_ms": 13.53004202246666, "decode_ms": 13.693084008991718, "receiver_prefill_ms": 140.44683298561722}, "total_wall_ms": 14304.402667097747, "ttft_ms": 13.53004202246666, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9822ffee8ba8f3eef61a70b67279ad855d4444cd4bb7438d0ca2685a028a22b8", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
57 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a2db1084287ce9ff31a6b850", "attempt_id": "81949e074967", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:19.578380Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 612, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.298665988259017, "decode_ms": 13.817040948197246, "receiver_prefill_ms": 73.64595902618021}, "total_wall_ms": 14280.350791988894, "ttft_ms": 13.298665988259017, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
58 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7323a45d69829abcf59a311b", "attempt_id": "1665b1c407f6", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:19.696449Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 615, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.854791992343962, "decode_ms": 133.00212495960295, "receiver_prefill_ms": 73.139916989021}, "total_wall_ms": 14399.512876057997, "ttft_ms": 13.854791992343962, "input_usage": {}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WM-90P-2460", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "3ddcc2bee71a4855b92c73846f02fa6f4653af8f0bfd70329622d6f1c31dccd5"}
59 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "db27f3905a2b89f687d3f767", "attempt_id": "0391283eb189", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:19.933925Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 620, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.558082981035113, "decode_ms": 93.36654201615602, "receiver_prefill_ms": 73.54254205711186}, "total_wall_ms": 14360.009291907772, "ttft_ms": 13.558082981035113, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3013170", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "a44e28276444065cad5bf760a512ece505a4d855205b9b2bb30a4479693b0a9f"}
60 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "3eaa0757a3c73e4332ea1578", "attempt_id": "367d099a0b02", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.131386Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 612, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.241624925285578, "decode_ms": 92.33474999200553, "receiver_prefill_ms": 73.24983295984566}, "total_wall_ms": 14358.351417002268, "ttft_ms": 13.241624925285578, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3039190", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "647471d76579dd2aa6dfa8f84e0a7b0abf733f02c7c2c9fe8239448f2768535e"}
61 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "43d9ce6ef4b176da18ed03cc", "attempt_id": "9c0df373bf58", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.327219Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 622, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.359667034819722, "decode_ms": 13.346542022190988, "receiver_prefill_ms": 73.31933407112956}, "total_wall_ms": 14279.622334055603, "ttft_ms": 13.359667034819722, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 1.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
62 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "afff9d86201cec57bd590874", "attempt_id": "bfa5aee9ea74", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.444275Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 616, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.358332915231586, "decode_ms": 92.88116602692753, "receiver_prefill_ms": 72.77325005270541}, "total_wall_ms": 14358.57991699595, "ttft_ms": 13.358332915231586, "input_usage": {}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "6201560", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "0d88693c5d70e48b97fd36a2779621a53e657b2d9bccd2889ca4c8b24c68806d"}
63 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "0de15f2f38440a8b76f539aa", "attempt_id": "2b6ed770ca2c", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.640350Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 616, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.727250043302774, "decode_ms": 13.356000068597496, "receiver_prefill_ms": 72.76795792859048}, "total_wall_ms": 14279.413167038001, "ttft_ms": 13.727250043302774, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
64 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "c469452b25b06069f9c119ba", "attempt_id": "804647b655bd", "run_group_id": "E5_smoke:merge_A5|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:20.757145Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "merge_A5", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512, "n_facts": 13}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 618, "message_tokens": 522, "message_utf8_bytes": 1340, "cache_tensor_bytes": null, "serialized_payload_bytes": 1340, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 14178.238791995682, "ttft_ms": 13.705165940336883, "decode_ms": 13.11649999115616, "receiver_prefill_ms": 73.18262499757111}, "total_wall_ms": 14279.700041981414, "ttft_ms": 13.705165940336883, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "f68a30f9a5d1e73e8872fb4924e200bc29a6951cb6092d0961578b9bfd6959ca", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
65 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "7847b741f00b4ce7970c227a", "attempt_id": "9f91842ac6a9", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:43.950387Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1376, "message_tokens": 1287, "message_utf8_bytes": 3340, "cache_tensor_bytes": null, "serialized_payload_bytes": 3340, "side_information_bytes": 0, "fetched_bytes": 1871, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 41.1847080104053, "decode_ms": 282.48854097910225, "receiver_prefill_ms": 331.38612494803965}, "total_wall_ms": 23733.73899993021, "ttft_ms": 41.1847080104053, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "1844550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ae95d76b4a91d3570273b15d8c20d44f5b348cc24d5dfdf7ae217730c17e2ab0", "output_hash": "23de9db5e2c3413d7142a9e77e743ed168ec6de65abf6413935059d107bb8c17"}
66 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "18d629a7d19c493521afdd59", "attempt_id": "40723f42711e", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:44.633980Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1338, "message_tokens": 1246, "message_utf8_bytes": 3244, "cache_tensor_bytes": null, "serialized_payload_bytes": 3244, "side_information_bytes": 0, "fetched_bytes": 1771, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 42.46612498536706, "decode_ms": 386.4938340848312, "receiver_prefill_ms": 330.83112502936274}, "total_wall_ms": 23837.171457940713, "ttft_ms": 42.46612498536706, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 10}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "WM-90P-2460", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "c903e22acc833959a3bbf0638fb1ee9fae27616194b965775737c0b7672960fb", "output_hash": "3ddcc2bee71a4855b92c73846f02fa6f4653af8f0bfd70329622d6f1c31dccd5"}
67 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "4ced35996da610c099012a59", "attempt_id": "820eb88fbc9e", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:45.416955Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1365, "message_tokens": 1268, "message_utf8_bytes": 3301, "cache_tensor_bytes": null, "serialized_payload_bytes": 3301, "side_information_bytes": 0, "fetched_bytes": 1832, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 19.3950830725953, "decode_ms": 337.39804197102785, "receiver_prefill_ms": 286.93091694731265}, "total_wall_ms": 23720.98304098472, "ttft_ms": 19.3950830725953, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3013170", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "fe2ea5528397251033d4174a5ce2849e153ab545799d9cb0d18c19e1f74399a5", "output_hash": "a44e28276444065cad5bf760a512ece505a4d855205b9b2bb30a4479693b0a9f"}
68 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "a9a8a0c4d432bb5ca8865856", "attempt_id": "0a718d636e3a", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:46.084217Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1355, "message_tokens": 1266, "message_utf8_bytes": 3293, "cache_tensor_bytes": null, "serialized_payload_bytes": 3293, "side_information_bytes": 0, "fetched_bytes": 1828, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 27.289082994684577, "decode_ms": 185.85829203948379, "receiver_prefill_ms": 375.7223749998957}, "total_wall_ms": 23666.18524992373, "ttft_ms": 27.289082994684577, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3039190", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "c6e87a60fb8104d185066ed693f734b701055dbd16ad4805501b55fce4a1c26a", "output_hash": "647471d76579dd2aa6dfa8f84e0a7b0abf733f02c7c2c9fe8239448f2768535e"}
69 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "5f42863ad361430b395ee32b", "attempt_id": "476d4bfd78d0", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:46.697210Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1374, "message_tokens": 1275, "message_utf8_bytes": 3285, "cache_tensor_bytes": null, "serialized_payload_bytes": 3285, "side_information_bytes": 0, "fetched_bytes": 1827, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 38.89429196715355, "decode_ms": 223.2152500655502, "receiver_prefill_ms": 352.5512919295579}, "total_wall_ms": 23691.884166910313, "ttft_ms": 38.89429196715355, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 6}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "Lithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "d49bf0d3d8d9b187ec63d83e81a8c5576985a6752461bd3bad41868126698f4e", "output_hash": "d2ace64e89bac774483fbdf2c3b630bf9c232f8f7b2aa06eda5f4a26d74f7cf0"}
70 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "e63b5a7f8f10a8640c22b3d3", "attempt_id": "4a99b3d331ad", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:47.335961Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1380, "message_tokens": 1287, "message_utf8_bytes": 3340, "cache_tensor_bytes": null, "serialized_payload_bytes": 3340, "side_information_bytes": 0, "fetched_bytes": 1871, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 20.951000042259693, "decode_ms": 286.4009999902919, "receiver_prefill_ms": 358.3094170317054}, "total_wall_ms": 23743.166374973953, "ttft_ms": 20.951000042259693, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 7}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "4403065", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "ae95d76b4a91d3570273b15d8c20d44f5b348cc24d5dfdf7ae217730c17e2ab0", "output_hash": "b06841909baef4e457116b1a8639e2707ad180e88be6a26a03e41f03f77fbb2b"}
71 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "920a8167c2783d722d0f0f92", "attempt_id": "33f897294bd5", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:48.023335Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1339, "message_tokens": 1246, "message_utf8_bytes": 3244, "cache_tensor_bytes": null, "serialized_payload_bytes": 3244, "side_information_bytes": 0, "fetched_bytes": 1771, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 42.38762496970594, "decode_ms": 35.475166980177164, "receiver_prefill_ms": 324.8105419334024}, "total_wall_ms": 23479.923208011314, "ttft_ms": 42.38762496970594, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "c903e22acc833959a3bbf0638fb1ee9fae27616194b965775737c0b7672960fb", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
72 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "c0005b4a22a9363b2616fef6", "attempt_id": "1181aba337a9", "run_group_id": "E5_smoke:shared_memory_A6|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:48.447224Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "shared_memory_A6", "implementation_fidelity": "custom", "compression_config": {"k": 2, "budget_tokens": 512}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 1350, "message_tokens": 1255, "message_utf8_bytes": 3257, "cache_tensor_bytes": null, "serialized_payload_bytes": 3257, "side_information_bytes": 0, "fetched_bytes": 1792, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 23074.44574998226, "ttft_ms": 37.491583032533526, "decode_ms": 48.52670803666115, "receiver_prefill_ms": 243.10816603247076}, "total_wall_ms": 23406.509208027273, "ttft_ms": 37.491583032533526, "input_usage": {"lookups": 6}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": "9b59bd103f78834d60cc7217f03f102d22542a7ee645009dcdac687299160ac4", "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
73 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "07f6e52a778e66864527904a", "attempt_id": "b43576eb2684", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:49.502726Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q04_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 41, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 98.5616659745574, "decode_ms": 590.0328749557957, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1401.089790975675, "ttft_ms": 98.5616659745574, "input_usage": {}, "output_usage": {"output_tokens": 16}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n1844550\n<|im_start|>robatni\nunknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 0.4}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "26ce51c96d59175d93d6596952c2cf98af15270047cff7dd5d0817be6651daf6"}
74 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "825a15b690efcce18cae054b", "attempt_id": "6c59b49831bb", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:50.221140Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q05_multi_hop", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 44, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 63.614208018407226, "decode_ms": 432.6625830726698, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1206.9736659759656, "ttft_ms": 63.614208018407226, "input_usage": {}, "output_usage": {"output_tokens": 13}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nYA-41S-3303", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "multi_hop"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "7063a3151528c78281542b673393a8cdd101b2d06b1e8adb0f19815928a53354"}
75 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "277cb9d37e93d596669dfb7d", "attempt_id": "9d1ef8ed7d31", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:50.743330Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q08_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 49, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 58.28074994497001, "decode_ms": 481.99387500062585, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1248.6895410111174, "ttft_ms": 58.28074994497001, "input_usage": {}, "output_usage": {"output_tokens": 12}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n1,844,550", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "c68142a24d688d117833ba79e64a8b34758a2d53fe8bc5337bc0d389dcbc80b5"}
76 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "5e9479fcaa84ed4182367340", "attempt_id": "e0c3d54d964e", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:51.305504Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q09_revision", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 41, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 49.72116707358509, "decode_ms": 365.5605419771746, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1123.2687489828095, "ttft_ms": 49.72116707358509, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "3,039,190", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "revision"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "f540ce7783d75dedddec953cb68a2b0f2472cd0a59b02bdb333d9cb537732b77"}
77 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "3e2f86abebf899aa1757af38", "attempt_id": "e67e1fe80273", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:51.743468Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q10_temporal", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 51, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 90.47212498262525, "decode_ms": 318.9096669666469, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1122.882082941942, "ttft_ms": 90.47212498262525, "input_usage": {}, "output_usage": {"output_tokens": 9}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\nLithban-10", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "temporal"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 0.0, "f1": 0.6666666666666666}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "92ab911faec640c58ff3f6e7d4d454de4d628c70c92cc85dd752f841f6aea1a6"}
78 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "74fb9e5eba23585fa3581ae4", "attempt_id": "d6ea887eb292", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:52.181156Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q11_aggregation", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 45, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 55.63445796724409, "decode_ms": 437.1588749345392, "receiver_prefill_ms": 0.0}, "total_wall_ms": 1200.9254159638658, "ttft_ms": 55.63445796724409, "input_usage": {}, "output_usage": {"output_tokens": 11}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "<|im_start|>user\n14290.42", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "aggregation"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 0.0, "abstained": 0.0, "f1": 0.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "c7569b7c33261a61449230bc152f178f7cc210ce3800ad959c0d4d3b98e4a541"}
79 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "b46f68e2943921254526a4cd", "attempt_id": "4a0e1f421332", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:52.697221Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q13_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 45, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 40.74966604821384, "decode_ms": 24.908291990868747, "receiver_prefill_ms": 0.0}, "total_wall_ms": 774.1670409450307, "ttft_ms": 40.74966604821384, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
80 +{"schema_version": "wp30-attempt-1", "study_id": "WP30", "experiment_id": "E5_smoke", "trial_id": "36c71d3273f01bddf02acad4", "attempt_id": "8c697cd8267f", "run_group_id": "E5_smoke:cache_concat_negative_control|k=2", "protocol_hash": "unfrozen", "config_hash": "1ce274e5b176df0f7757882585550b4686e03aee57958a30a6868cf39820b24d", "git_commit": "fea8e12394fac49c22b00e41baf681fd35f0576e", "timestamp_utc": "2026-10-30T03:36:52.786717Z", "task_family": "synthetic", "document_id": "syn_02003_2000", "document_hash": "c88d189ebb79e2f792c48ae6111a7e7bf78acb1c3c41f7ea40312f6e47e3dbd8", "question_id": "q14_unanswerable", "split": "dev", "cluster_id": "syn_02003_2000", "sender_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "receiver_model_revision": "mlx-community/Qwen2.5-3B-Instruct-bf16@bec1ad658e380044bbfe06d4a2268801d3379a11", "tokenizer_hashes": {"receiver": "5c5a439ba2677fe28e1dea9b22371b270ecd5e34acbc849bc619689605f9547b"}, "backend_versions": {"mlx": "0.32.4", "mlx_lm": "0.32.2"}, "hardware_id": "laptop-m5max-48g", "device_state": {}, "weight_precision": "bf16", "method_id": "cache_concat_negative_control", "implementation_fidelity": "n/a", "compression_config": {"k": 2, "budget_tokens": 512, "cache_bytes": 88584192}, "question_access": "blind", "topology": "fanin_k2", "handoff_index": 1, "reuse_count": 8, "cache_hit": false, "cache_compatibility_hash": null, "random_seed": 0, "source_tokens_sender": null, "source_tokens_receiver": 1989, "query_tokens": 47, "message_tokens": null, "message_utf8_bytes": 8282, "cache_tensor_bytes": 88584192, "serialized_payload_bytes": 88584192, "side_information_bytes": 0, "fetched_bytes": 0, "transport_bytes_if_measured": null, "timing_spans": {"build_ms": 705.7535409694538, "ttft_ms": 68.34420806262642, "decode_ms": 22.9576249839738, "receiver_prefill_ms": 0.0}, "total_wall_ms": 800.9481240296736, "ttft_ms": 68.34420806262642, "input_usage": {}, "output_usage": {"output_tokens": 1}, "provider_cached_usage": null, "process_memory": null, "backend_memory": null, "peak_prefill_memory": null, "peak_decode_memory": null, "mapper_memory": null, "cost_with_source": null, "prediction_artifact": "unknown", "evidence_artifact": {"conflict": {"code": "Ripra-10", "bogus_amount": "1,449,880", "source": "S2"}, "question_type": "unanswerable"}, "reference_key_id": "keys/synthetic/syn_02003_2000.json", "task_scores": {"correct": 1.0, "abstained": 1.0, "f1": 1.0}, "status": "success", "error_code": null, "retry_of": null, "log_artifact": "runs/logs/E5_smoke.log", "payload_hash": null, "output_hash": "b23a6a8439c0dde5515893e7c90c1e3233b8616e634470f20dc4928bcf3609bc"}
added runs/raw/chains/E4_smoke_syn_02000_2000_A4_closed_qwen3b.json +74 −0
@@ -0,0 +1,74 @@
1 +{
2 + "hops": {
3 + "1": "**Summary:**\n\n- **Capital Projects Register, Issue 165**\n - **Proposals under review:** Zennovvol-7, Ruktor-14, Torsel-10 (no attributes recorded)\n - **Project Zennovvol-6:**\n - Proposed by Idris Abubakar on 5 Nov 2033\n - Initial budget: 6,889,172 NOK\n - Revised budget: 7,280,585 NOK\n - Approved on 8 Mar 2034\n - Site: Lot 208-C, Senneterre\n - Supplier: BH-71Y-2851\n - Launch postponed to 22 Apr 2034\n - Rated capacity: 35.2 km\n - Calibration constant: 58.0820\n - **Project Ruktor-10:**\n - Proposed by Teodora Mäkelä on 2 Jul 2032\n - Initial budget: 5,869,675 NOK\n - Revised budget: 5,802,310 NOK\n - Approved on 27 Oct 2032\n - Site: Lot 50-D, Matagami\n - Supplier: SP-38G-4217\n - Cancelled\n - Rated capacity: 30.99 MW\n - Calibration constant: 54.0816\n - **Project Torsel-5:**\n - Proposed by Haruki Rousseau on 18 Apr 2031\n - Initial budget: 1,661,075 CAD\n - Revised budget: 1,945,685 CAD\n - Approved on 13 Sep 2031\n - Site: Lot 835-P, Fermont\n - Supplier: WG-14H-1182\n - On schedule\n - Rated capacity: 7.64 MW\n - Calibration constant: 68.6281\n - **Project Eshtal-12:**\n - Proposed by Oksana Okonkwo on 23",
4 + "2": "**Summary:**\n- **Capital Projects Register, Issue 165**\n - **Proposals under review:** Zennovvol-7, Ruktor-14, Torsel-10 (no attributes recorded)\n - **Project Zennovvol-6:**\n - Proposed by Idris Abubakar on 5 Nov 2033\n - Initial budget: 6,889,172 NOK\n - Revised budget: 7,280,585 NOK\n - Approved on 8 Mar 2034\n - Site: Lot 208-C, Senneterre\n - Supplier: BH-71Y-2851\n - Launch postponed to 22 Apr 2034\n - Rated capacity: 35.2 km\n - Calibration constant: 58.0820\n - **Project Ruktor-10:**\n - Proposed by Teodora Mäkelä on 2 Jul 2032\n - Initial budget: 5,869,675 NOK\n - Revised budget: 5,802,310 NOK\n - Approved on 27 Oct 2032\n - Site: Lot 50-D, Matagami\n - Supplier: SP-38G-4217\n - Cancelled\n - Rated capacity: 30.99 MW\n - Calibration constant: 54.0816\n - **Project Torsel-5:**\n - Proposed by Haruki Rousseau on 18 Apr 2031\n - Initial budget: 1,661,075 CAD\n - Revised budget: 1,945,685 CAD\n - Approved on 13 Sep 2031\n - Site: Lot 835-P, Fermont\n - Supplier: WG-14H-1182\n - On schedule\n - Rated capacity: 7.64 MW\n - Calibration constant: 68.6281\n - **Project Eshtal-12:** Proposed by Oksana Okonkwo on 23"
5 + },
6 + "meta": {
7 + "1": {
8 + "hop_ms": 7388.310500071384,
9 + "agent": "mlx-community/Qwen2.5-3B-Instruct-bf16",
10 + "usage": {
11 + "sender_input_tokens": 2087,
12 + "sender_output_tokens": 512,
13 + "sender_prefill_ms": 258.550624945201,
14 + "sender_ttft_ms": 14.011999941430986,
15 + "sender_decode_ms": 7108.514915918931
16 + },
17 + "repair": {
18 + "lookups": 0,
19 + "fetched_bytes": 0
20 + },
21 + "ledger": {
22 + "facts_total": 64,
23 + "facts_retained": 28,
24 + "retention_rate": 0.4375,
25 + "retention_by_kind": {
26 + "person": 0.6666666666666666,
27 + "date": 0.0,
28 + "amount": 0.42857142857142855,
29 + "status": 0.75,
30 + "place": 0.0,
31 + "identifier": 0.5,
32 + "quantity": 0.5
33 + },
34 + "unsupported_numbers": 0,
35 + "unsupported_identifiers": 0,
36 + "distorted_sentences": 0,
37 + "message_bytes": 1196
38 + }
39 + },
40 + "2": {
41 + "hop_ms": 7029.634707956575,
42 + "agent": "mlx-community/Qwen2.5-3B-Instruct-bf16",
43 + "usage": {
44 + "sender_input_tokens": 629,
45 + "sender_output_tokens": 509,
46 + "sender_prefill_ms": 86.81391598656774,
47 + "sender_ttft_ms": 13.69220798369497,
48 + "sender_decode_ms": 6928.4298330312595
49 + },
50 + "repair": {
51 + "lookups": 0,
52 + "fetched_bytes": 0
53 + },
54 + "ledger": {
55 + "facts_total": 64,
56 + "facts_retained": 28,
57 + "retention_rate": 0.4375,
58 + "retention_by_kind": {
59 + "person": 0.6666666666666666,
60 + "date": 0.0,
61 + "amount": 0.42857142857142855,
62 + "status": 0.75,
63 + "place": 0.0,
64 + "identifier": 0.5,
65 + "quantity": 0.5
66 + },
67 + "unsupported_numbers": 0,
68 + "unsupported_identifiers": 0,
69 + "distorted_sentences": 0,
70 + "message_bytes": 1189
71 + }
72 + }
73 + }
74 +}
\ No newline at end of file
added runs/raw/chains/E4_smoke_syn_02000_2000_A5_repairable_qwen3b_llama3b.json +75 −0
@@ -0,0 +1,75 @@
1 +{
2 + "hops": {
3 + "1": "{\"facts\": [{\"entity\": \"Zennovvol-6\", \"attribute\": \"proposed by\", \"value\": \"Idris Abubakar\", \"span\": [238, 321]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"initial budget\", \"value\": \"6,889,172 NOK\", \"span\": [323, 433]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"final budget\", \"value\": \"7,280,585 NOK\", \"span\": [434, 508]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"approval date\", \"value\": \"8 March 2034\", \"span\": [509, 576]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"launch date\", \"value\": \"22 April 2034\", \"span\": null}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"rated capacity\", \"value\": \"35.2 km\", \"span\": [759, 804]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"supplier identifier\", \"value\": \"BH-71Y-2851\", \"span\": [629, 684]}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"status\", \"value\": \"NOT cancelled\", \"span\": [685, 758]}]}",
4 + "2": "{\"facts\": [{\"entity\": \"Zennovvol-6\", \"attribute\": \"proposed by\", \"value\": \"Idris Abubakar\", \"quote\": \"Project Zennovvol-6 was proposed by Idris Abubakar on 5 November 2033 with an initial budget of 6,889,172 NOK.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"initial budget\", \"value\": \"6,889,172 NOK\", \"quote\": \"Project Zennovvol-6 was proposed by Idris Abubakar on 5 November 2033 with an initial budget of 6,889,172 NOK.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"final budget\", \"value\": \"7,280,585 NOK\", \"quote\": \"On 19 January 2034 the budget of Zennovvol-6 was revised to 7,280,585 NOK.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"approval date\", \"value\": \"8 March 2034\", \"quote\": \"The approved decision for Zennovvol-6 was recorded on 8 March 2034.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"launch date\", \"value\": \"22 April 2034\", \"quote\": \"Zennovvol-6 was NOT cancelled; its launch was postponed to 22 April 2034.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"rated capacity\", \"value\": \"35.2 km\", \"quote\": \"The rated capacity of Zennovvol-6 is 35.2 km.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"supplier identifier\", \"value\": \"BH-71Y-2851\", \"quote\": \"The supplier identifier for Zennovvol-6 is BH-71Y-2851.\"}, {\"entity\": \"Zennovvol-6\", \"attribute\": \"status\", \"value\": \"NOT cancelled\", \"quote\": \"Zennovvol-6 was NOT cancelled; its launch was postponed to 22 April 2034.\"}], \"notes\": \"\", \"unknowns\": []}"
5 + },
6 + "meta": {
7 + "1": {
8 + "hop_ms": 7189.094333909452,
9 + "agent": "mlx-community/Qwen2.5-3B-Instruct-bf16",
10 + "usage": {
11 + "sender_input_tokens": 2115,
12 + "sender_output_tokens": 512,
13 + "sender_prefill_ms": 247.3138750065118,
14 + "sender_ttft_ms": 13.779832981526852,
15 + "sender_decode_ms": 6926.056583062746
16 + },
17 + "repair": {
18 + "lookups": 0,
19 + "fetched_bytes": 0
20 + },
21 + "ledger": {
22 + "facts_total": 64,
23 + "facts_retained": 10,
24 + "retention_rate": 0.15625,
25 + "retention_by_kind": {
26 + "person": 0.16666666666666666,
27 + "date": 0.125,
28 + "amount": 0.14285714285714285,
29 + "status": 0.3333333333333333,
30 + "place": 0.0,
31 + "identifier": 0.08333333333333333,
32 + "quantity": 0.16666666666666666
33 + },
34 + "unsupported_numbers": 0,
35 + "unsupported_identifiers": 0,
36 + "distorted_sentences": 0,
37 + "message_bytes": 817
38 + }
39 + },
40 + "2": {
41 + "hop_ms": 7899.107125005685,
42 + "agent": "mlx-community/Llama-3.2-3B-Instruct-bf16",
43 + "usage": {
44 + "sender_input_tokens": 709,
45 + "sender_output_tokens": 492,
46 + "sender_prefill_ms": 825.0708330888301,
47 + "sender_ttft_ms": 37.18812495935708,
48 + "sender_decode_ms": 7024.092499981634
49 + },
50 + "repair": {
51 + "lookups": 1,
52 + "fetched_bytes": 540,
53 + "codes_mentioned": 1
54 + },
55 + "ledger": {
56 + "facts_total": 64,
57 + "facts_retained": 16,
58 + "retention_rate": 0.25,
59 + "retention_by_kind": {
60 + "person": 0.16666666666666666,
61 + "date": 0.25,
62 + "amount": 0.14285714285714285,
63 + "status": 0.75,
64 + "place": 0.0,
65 + "identifier": 0.08333333333333333,
66 + "quantity": 0.16666666666666666
67 + },
68 + "unsupported_numbers": 0,
69 + "unsupported_identifiers": 0,
70 + "distorted_sentences": 0,
71 + "message_bytes": 1403
72 + }
73 + }
74 + }
75 +}
\ No newline at end of file
added scripts/e6_network.py +76 −0
@@ -0,0 +1,76 @@
1 +"""E6/E2: genuine two-node cache transfer over the LAN (laptop -> M3U96b worker) vs text transfer vs local recompute.
2 +Usage: python scripts/e6_network.py --host 192.168.2.85 --port 9430 --model qwen3b --docs 6 --lengths 2000 8000 16000 --repeats 3
3 +Requires the worker: on the remote node `python -m wp30 worker --port 9430 --model <repo>` (same checkpoint revision).
4 +Loopback mode (--host 127.0.0.1) is labelled 'loopback' and is NOT a network measurement."""
5 +import argparse, json, time, statistics, subprocess, sys, socket
6 +import yaml, numpy as np, mlx.core as mx
7 +from wp30.util import ROOT, read_json, utc_now, append_jsonl, sha256_text
8 +from wp30.data.prepare import load_doc, load_key
9 +from wp30.backends.mlx_backend import MLXModel
10 +from wp30.cache.sparse import quantize_cache, cache_tensor_bytes, to_sparse_all
11 +from wp30.cache.serialize import build_manifest, serialize_cache, codec_compress
12 +from wp30.transport import Client
13 +from wp30.prompts import SYSTEM_RECEIVER, receiver_user_text
14 +from wp30.scoring import score_synthetic
15 +
16 +ap = argparse.ArgumentParser()
17 +ap.add_argument("--host", required=True); ap.add_argument("--port", type=int, default=9430); ap.add_argument("--model", default="qwen3b")
18 +ap.add_argument("--docs", type=int, default=6); ap.add_argument("--lengths", type=int, nargs="+", default=[2000, 8000, 16000]); ap.add_argument("--repeats", type=int, default=3)
19 +ap.add_argument("--label", default=None); ap.add_argument("--split", default="dev"); ap.add_argument("--out", default="runs/raw/E6_network.jsonl")
20 +a = ap.parse_args()
21 +label = a.label or ("loopback" if a.host.startswith("127.") else "lan")
22 +mcfg = yaml.safe_load(open(ROOT / "configs/models.yaml"))["models"]
23 +m = MLXModel(mcfg[a.model]["repo"])
24 +cl = Client(a.host, a.port)
25 +h, _, meta = cl.call({"cmd": "ping"})
26 +assert h["config_hash"] == m.info.config_hash, "remote model differs"
27 +print("remote ok", h["model_revision"], "rtt_ms", round(meta["roundtrip_ms"], 2))
28 +# raw link bandwidth probe with an incompressible 64 MiB payload (frame echo is not implemented; use run_text no-op? -> use a dedicated ping with payload)
29 +man = read_json(ROOT / "data/manifests/splits.json")
30 +items = [it for it in man["items"] if it["family"] == "synthetic" and it["split"] == a.split and it.get("target_tokens") in a.lengths]
31 +by_len = {L: [it for it in items if it["target_tokens"] == L][: a.docs] for L in a.lengths}
32 +out = ROOT / a.out
33 +for L, its in by_len.items():
34 + for it in its:
35 + doc = load_doc(it["document_id"], "synthetic"); key = load_key(it["document_id"], "synthetic")
36 + q = doc["questions"][0]; qk = key["questions"][0]
37 + full = m.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text("Document", doc["text"], q["question"], "synthetic")}])
38 + pre_txt = m.tokenizer.apply_chat_template([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": "Document:\n" + doc["text"] + "\n\n"}], add_generation_prompt=False, tokenize=False)
39 + pre = m.encode(pre_txt); n = 0
40 + while n < min(len(pre), len(full)) and pre[n] == full[n]: n += 1
41 + pre, query = full[:n], full[n:]
42 + for rep in range(a.repeats):
43 + # (1) local sender prefill + serialize + transfer cache + remote load + remote query [warm remote cache scenario: cache moved]
44 + for variant in [("B1", 8 if False else None, "none"), ("B1", None, "zstd"), ("B2", 8, "none"), ("B2", 4, "none")]:
45 + meth, bits, codec = variant
46 + t0 = time.perf_counter(); c, tp = m.prefill(pre)
47 + c2 = to_sparse_all(c) if bits is None else quantize_cache(c, bits)
48 + mx.eval([x for cc in c2 for x in (cc.keys if isinstance(cc.keys, tuple) else (cc.keys,))]); mx.synchronize()
49 + t1 = time.perf_counter()
50 + manf = build_manifest(m.info, pre, c2, it["document_id"], meth, {"bits": bits})
51 + payload = serialize_cache(c2, manf); t2 = time.perf_counter()
52 + comp, cms = codec_compress(payload, codec) if codec != "none" else (payload, 0.0)
53 + hdr = {"cmd": "run_cache", "codec": codec, "prefix_hash": manf["prefix_token_hash"], "query_ids": query, "max_tokens": 32}
54 + rh, _, tm = cl.call(hdr, comp)
55 + rec = {"experiment_id": "E6_network", "label": label, "timestamp_utc": utc_now(), "document_id": it["document_id"], "target_tokens": L, "prefix_tokens": len(pre), "query_tokens": len(query), "repeat": rep,
56 + "method": meth, "bits": bits, "codec": codec, "status": rh.get("status"), "answer": rh.get("text"), "correct": score_synthetic(rh.get("text", ""), qk)["correct"] if rh.get("text") else None,
57 + "cache_tensor_bytes": cache_tensor_bytes(c2), "serialized_bytes": len(payload), "transport_bytes": tm["transport_bytes"], "sender_prefill_ms": tp["prefill_ms"], "compress_ms": (t1 - t0) * 1000 - tp["prefill_ms"],
58 + "serialize_ms": (t2 - t1) * 1000, "codec_ms": cms, "send_ms": tm["send_ms"], "roundtrip_ms": tm["roundtrip_ms"], "remote_recv_ms": rh.get("recv_ms"), "remote_decompress_ms": rh.get("decompress_ms"),
59 + "remote_load_ms": rh.get("load_ms"), "remote_ttft_ms": rh.get("ttft_ms"), "remote_decode_ms": rh.get("decode_ms"), "remote_server_ms": rh.get("server_ms"), "remote_peak_bytes": rh.get("peak_bytes"),
60 + "effective_gbps": (tm["transport_bytes"] * 8 / 1e9) / max(1e-9, (tm["roundtrip_ms"] - (rh.get("server_ms") or 0)) / 1000)}
61 + append_jsonl(out, rec)
62 + print(f"{label} L={L} {meth} bits={bits} codec={codec} rep={rep} bytes={len(comp):>10d} send={tm['send_ms']:7.1f}ms rt={tm['roundtrip_ms']:7.1f}ms remote_load={rh.get('load_ms',0):6.1f} ttft={rh.get('ttft_ms',0):6.1f} eff={rec['effective_gbps']:.2f}Gb/s corr={rec['correct']}")
63 + # (2) text transfer: send token ids, remote does full prefill [cold remote scenario]
64 + rh, _, tm = cl.call({"cmd": "run_text", "token_ids": full, "max_tokens": 32})
65 + rec = {"experiment_id": "E6_network", "label": label, "timestamp_utc": utc_now(), "document_id": it["document_id"], "target_tokens": L, "prefix_tokens": len(pre), "query_tokens": len(query), "repeat": rep,
66 + "method": "text_remote_prefill", "bits": None, "codec": "none", "status": "success", "answer": rh.get("text"), "correct": score_synthetic(rh.get("text", ""), qk)["correct"],
67 + "serialized_bytes": len(doc["text"].encode()), "transport_bytes": tm["transport_bytes"], "send_ms": tm["send_ms"], "roundtrip_ms": tm["roundtrip_ms"], "remote_prefill_ms": rh.get("prefill_ms"),
68 + "remote_ttft_ms": rh.get("ttft_ms"), "remote_decode_ms": rh.get("decode_ms"), "remote_server_ms": rh.get("server_ms"), "remote_peak_bytes": rh.get("peak_bytes")}
69 + append_jsonl(out, rec)
70 + print(f"{label} L={L} text_remote rep={rep} bytes={tm['transport_bytes']:>10d} rt={tm['roundtrip_ms']:7.1f}ms remote_prefill={rh.get('prefill_ms',0):7.1f} ttft={rh.get('ttft_ms',0):6.1f}")
71 + # (3) local processing (no transfer): run where the data is
72 + c, tp = m.prefill(pre); g = m.generate_from_cache(c, query, max_tokens=32)
73 + append_jsonl(out, {"experiment_id": "E6_network", "label": "local", "timestamp_utc": utc_now(), "document_id": it["document_id"], "target_tokens": L, "prefix_tokens": len(pre), "query_tokens": len(query), "repeat": rep,
74 + "method": "local_prefill", "status": "success", "answer": g["text"], "correct": score_synthetic(g["text"], qk)["correct"], "local_prefill_ms": tp["prefill_ms"], "local_ttft_ms": g["ttft_ms"], "local_decode_ms": g["decode_ms"]})
75 +cl.close()
76 +print("done ->", out)
modified src/wp30/backends/mlx_backend.py +10 −1
@@ -160,6 +160,15 @@ class MLXModel:
160 160 out_ids: list[int] = []
161 161 t.start("query_prefill")
162 162 toks = mx.array(query_ids, dtype=mx.int32)
163 + # Heterogeneous retained lengths across layers (e.g. PyramidKV budgets) cannot share one attention mask
164 + # (mlx_lm builds the mask from cache[0]); process the query tokens one at a time in that case (mask=None).
165 + kept = {getattr(c, "n_kept", None) for c in cache}
166 + heterogeneous = len(kept) > 1
167 + if heterogeneous and len(query_ids) > 1:
168 + for i in range(len(query_ids) - 1):
169 + self.model(toks[i:i + 1][None], cache=cache)
170 + mx.eval([c.keys for c in cache])
171 + toks = toks[-1:]
163 172 logits = self.model(toks[None], cache=cache)[:, -1, :]
164 173 logprobs = logits - mx.logsumexp(logits, keepdims=True)
165 174 y = sampler(logprobs)
@@ -186,7 +195,7 @@ class MLXModel:
186 195 mx.synchronize()
187 196 t.stop("decode")
188 197 text = self.decode(out_ids)
189 − return {"text": text, "output_ids": out_ids, "output_tokens": len(out_ids), "ttft_ms": ttft_ms,
198 + return {"text": text, "output_ids": out_ids, "output_tokens": len(out_ids), "ttft_ms": ttft_ms, "sequential_query_prefill": heterogeneous,
190 199 "query_prefill_ms": t.spans["query_prefill"], "decode_ms": t.spans["decode"], "query_tokens": len(query_ids)}
191 200
192 201 def generate_text(self, token_ids: list[int], max_tokens: int = 64, temp: float = 0.0) -> dict:
added src/wp30/chains.py +159 −0
@@ -0,0 +1,159 @@
1 +"""E4: repeated handoffs (relay chains) with an evaluator-only fact ledger.
2 +
3 +Chain: agent_1 reads the document and writes m_1 (A4 prose or A5 structured, question-blind). agent_i (i>1) reads only
4 +m_{i-1} (closed) or m_{i-1} plus metered source lookups (repairable) and writes m_i under the same budget. After hops
5 +h in HOPS the receiver answers all questions from m_h. Retention/distortion is scored from the ledger (never shown to agents).
6 +"""
7 +from __future__ import annotations
8 +import json, re, time, yaml
9 +from pathlib import Path
10 +import mlx.core as mx
11 +from .util import ROOT, read_json, sha256_json, sha256_text, utc_now, read_jsonl
12 +from .records import new_record, write_record, trial_key
13 +from .data.prepare import load_doc, load_key
14 +from .scoring import score_synthetic, normalize_answer, extract_numbers
15 +from .prompts import SYSTEM_RECEIVER, receiver_user_text, SUMMARY_SYSTEM, EVIDENCE_SYSTEM
16 +from .backends.mlx_backend import MLXModel, reset_peak
17 +from .methods import text as TM
18 +
19 +HARDWARE_ID = "laptop-m5max-48g"
20 +
21 +RELAY_SUMMARY_USER = ("You received the following notes from a colleague about a document you cannot see. Rewrite them as a faithful summary of at most "
22 + "{budget} tokens for the next colleague, preserving every concrete fact (names, identifiers, numbers, units, dates, statuses, negations) "
23 + "exactly as written. Do not add facts that are not in the notes.{repair}\n\nNotes:\n{msg}\n\nSummary:")
24 +RELAY_EVIDENCE_USER = ("You received the following structured evidence (JSON) from a colleague about a document you cannot see. Re-emit it as a JSON object with the same "
25 + "schema (keys facts, optional notes, optional unknowns), under {budget} tokens, keeping every fact and value exactly; drop nothing unless the "
26 + "budget forces you to, and never invent values.{repair}\n\nEvidence:\n{msg}\n\nJSON:")
27 +
28 +def _ledger_metrics(message: str, facts: list[dict], doc_text: str) -> dict:
29 + """Retention: a fact counts as retained if its value (normalized) appears in the message. Unsupported additions:
30 + numbers with >=4 significant digits, identifiers (XX-00X-0000) and 4-digit-year dates in the message that do not
31 + occur in the document."""
32 + msg_n = normalize_answer(message); msg_nc = message.replace(",", "")
33 + retained = 0; by_kind = {}
34 + for f in facts:
35 + v = f["value"]
36 + ok = normalize_answer(v) in msg_n or v.replace(",", "") in msg_nc
37 + by_kind.setdefault(f["kind"], [0, 0]); by_kind[f["kind"]][1] += 1
38 + if ok:
39 + retained += 1; by_kind[f["kind"]][0] += 1
40 + doc_nums = {str(int(x)) for x in extract_numbers(doc_text) if abs(x) >= 1000}
41 + msg_nums = {str(int(x)) for x in extract_numbers(message) if abs(x) >= 1000}
42 + unsupported_nums = msg_nums - doc_nums
43 + doc_ids = set(re.findall(r"\b[A-Z]{2}-\d{2}[A-Z]-\d{4}\b", doc_text)); msg_ids = set(re.findall(r"\b[A-Z]{2}-\d{2}[A-Z]-\d{4}\b", message))
44 + unsupported_ids = msg_ids - doc_ids
45 + # distorted amounts: a project code co-occurring (same sentence) with an unsupported number
46 + distorted = 0
47 + for sent in re.split(r"(?<=[.!?])\s+|\n+", message):
48 + if re.search(r"\b[A-Z][a-z]+-\d{1,2}\b", sent) and any(n in sent.replace(",", "") for n in unsupported_nums):
49 + distorted += 1
50 + return {"facts_total": len(facts), "facts_retained": retained, "retention_rate": retained / max(1, len(facts)),
51 + "retention_by_kind": {k: v[0] / v[1] for k, v in by_kind.items()},
52 + "unsupported_numbers": len(unsupported_nums), "unsupported_identifiers": len(unsupported_ids), "distorted_sentences": distorted,
53 + "message_bytes": len(message.encode("utf-8"))}
54 +
55 +def _repair_lookup(prev_msg: str, doc_text: str, max_bytes: int, max_lookups: int) -> tuple[str, dict]:
56 + """Repairable chain: the agent re-reads source sentences for entity codes mentioned in the notes (metered)."""
57 + codes = []
58 + for c in re.findall(r"\b[A-Z][a-z]+-\d{1,2}\b", prev_msg):
59 + if c not in codes: codes.append(c)
60 + sents = re.split(r"(?<=[.!?])\s+", doc_text)
61 + fetched, nb, n = [], 0, 0
62 + for c in codes:
63 + if n >= max_lookups: break
64 + hits = [s for s in sents if c in s]
65 + if not hits: continue
66 + chunk = " ".join(hits)
67 + b = len(chunk.encode("utf-8"))
68 + if nb + b > max_bytes: continue
69 + fetched.append(f"[source passages for {c}] {chunk}"); nb += b; n += 1
70 + return "\n".join(fetched), {"lookups": n, "fetched_bytes": nb, "codes_mentioned": len(codes)}
71 +
72 +def run_chains(config_path: str, resume: bool = False) -> int:
73 + cfg = yaml.safe_load(open(config_path))
74 + mcfg = yaml.safe_load(open(ROOT / "configs/models.yaml"))["models"]
75 + models = {}
76 + def get(a):
77 + if a not in models: models[a] = MLXModel(mcfg[a]["repo"])
78 + return models[a]
79 + man = read_json(ROOT / "data/manifests/splits.json")
80 + items = [it for it in man["items"] if it["family"] == "synthetic" and it["split"] in cfg["splits"] and it.get("target_tokens") in cfg["lengths"]]
81 + if cfg.get("limit_docs"): items = items[: cfg["limit_docs"]]
82 + out_path = ROOT / "runs/raw" / f"{cfg['experiment_id']}.jsonl"
83 + log_path = ROOT / "runs/logs" / f"{cfg['experiment_id']}.log"
84 + done = {r["trial_id"] for r in read_jsonl(out_path) if r["status"] == "success"} if (resume and out_path.exists()) else set()
85 + protocol_hash = cfg.get("protocol_hash", "unfrozen"); config_hash = sha256_json(cfg)
86 + def log(m):
87 + line = f"{utc_now()} {m}"; print(line, flush=True)
88 + with open(log_path, "a") as f: f.write(line + "\n")
89 + log(f"chains start {cfg['experiment_id']} items={len(items)} done={len(done)}")
90 + HOPS = cfg["hops"]; budget = cfg["budget_tokens"]
91 + for it in items:
92 + doc = load_doc(it["document_id"], "synthetic"); key = load_key(it["document_id"], "synthetic")
93 + qkeys = {q["question_id"]: q for q in key["questions"]}
94 + for chain in cfg["chains"]:
95 + rep, mode, agents = chain["representation"], chain["mode"], chain["agents"] # agents: list of aliases cycled
96 + cid = f"{rep}|{mode}|{'-'.join(agents)}"
97 + # skip if all hop trials done
98 + keys_needed = [trial_key(cfg["experiment_id"], it["document_id"], q["question_id"], rep, {"mode": mode, "agents": agents, "budget": budget}, agents[0], agents[-1], "blind", protocol_hash, 0, "chain", h) for h in HOPS for q in doc["questions"]]
99 + if all(k in done for k in keys_needed):
100 + continue
101 + msg = None; hop_msgs = {}; hop_meta = {}
102 + t_chain = 0.0; fetched_total = 0; lookups_total = 0
103 + for h in range(1, max(HOPS) + 1):
104 + agent = get(agents[(h - 1) % len(agents)])
105 + t0 = time.perf_counter()
106 + repair_txt, rinfo = "", {"lookups": 0, "fetched_bytes": 0}
107 + if h == 1:
108 + m = TM.a4_summary(doc["text"], budget, agent, None) if rep == "A4" else TM.a5_structured(doc["text"], budget, agent, None)
109 + text = m.text; usage = m.sender_usage
110 + else:
111 + if mode == "repairable":
112 + repair_txt, rinfo = _repair_lookup(msg, doc["text"], cfg.get("max_fetch_bytes", 1500), cfg.get("max_lookups", 4))
113 + rep_clause = ("\n\nYou may also use these passages re-read from the original source:\n" + repair_txt) if repair_txt else ""
114 + if rep == "A4":
115 + user = RELAY_SUMMARY_USER.format(budget=budget, msg=msg, repair=rep_clause)
116 + text, usage, _ = TM._sender_generate(agent, SUMMARY_SYSTEM, user, budget)
117 + else:
118 + user = RELAY_EVIDENCE_USER.format(budget=budget, msg=msg, repair=rep_clause)
119 + raw, usage, _ = TM._sender_generate(agent, EVIDENCE_SYSTEM, user, budget)
120 + obj = TM._parse_json_loose(raw)
121 + text = json.dumps(obj, ensure_ascii=False) if isinstance(obj, dict) else raw
122 + hop_ms = (time.perf_counter() - t0) * 1000; t_chain += hop_ms
123 + fetched_total += rinfo["fetched_bytes"]; lookups_total += rinfo["lookups"]
124 + msg = text; hop_msgs[h] = text
125 + hop_meta[h] = {"hop_ms": hop_ms, "agent": agent.info.repo_id, "usage": usage, "repair": rinfo, "ledger": _ledger_metrics(text, key["facts"], doc["text"])}
126 + if h in HOPS:
127 + receiver = get(agents[(h - 1) % len(agents)]) # the last agent in the chain answers
128 + for q in doc["questions"]:
129 + tk = trial_key(cfg["experiment_id"], it["document_id"], q["question_id"], rep, {"mode": mode, "agents": agents, "budget": budget}, agents[0], agents[-1], "blind", protocol_hash, 0, "chain", h)
130 + if tk in done: continue
131 + rec = new_record(experiment_id=cfg["experiment_id"], trial_id=tk, run_group_id=f"{cfg['experiment_id']}:{cid}:h{h}", protocol_hash=protocol_hash, config_hash=config_hash,
132 + task_family="synthetic", document_id=it["document_id"], document_hash=it["document_hash"], question_id=q["question_id"], split=it["split"], cluster_id=it["cluster_id"],
133 + sender_model_revision="|".join(f"{get(a).info.repo_id}@{get(a).info.revision}" for a in agents), receiver_model_revision=f"{receiver.info.repo_id}@{receiver.info.revision}",
134 + tokenizer_hashes={"receiver": receiver.info.tokenizer_hash}, backend_versions={"mlx": mx.__version__, "mlx_lm": receiver.info.mlx_lm_version}, hardware_id=HARDWARE_ID,
135 + device_state={}, weight_precision="bf16", method_id=rep, implementation_fidelity="custom", compression_config={"mode": mode, "agents": agents, "budget_tokens": budget},
136 + question_access="blind", topology=f"chain_{mode}", handoff_index=h, reuse_count=len(doc["questions"]), cache_hit=False, cache_compatibility_hash=None, random_seed=0,
137 + source_tokens_sender=None, source_tokens_receiver=len(receiver.encode(doc["text"])), message_utf8_bytes=len(text.encode("utf-8")), serialized_payload_bytes=len(text.encode("utf-8")),
138 + cache_tensor_bytes=None, side_information_bytes=0, fetched_bytes=fetched_total, timing_spans={"chain_ms": t_chain, "hops": {str(k): v["hop_ms"] for k, v in hop_meta.items()}},
139 + input_usage={"lookups": lookups_total}, output_usage={}, provider_cached_usage=None, process_memory=None, backend_memory=None, peak_prefill_memory=None, peak_decode_memory=None,
140 + mapper_memory=None, cost_with_source=None, evidence_artifact={"ledger": hop_meta[h]["ledger"], "ledger_by_hop": {str(k): v["ledger"] for k, v in hop_meta.items()}},
141 + reference_key_id=f"keys/synthetic/{it['document_id']}.json", status="implementation_error", error_code=None, log_artifact=str(log_path.relative_to(ROOT)), payload_hash=sha256_text(text), output_hash=None)
142 + try:
143 + ids = receiver.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text("Notes received after %d handoff(s)" % h, text, q["question"], "synthetic")}])
144 + t1 = time.perf_counter(); g = receiver.generate_text(ids, max_tokens=48)
145 + rec.update({"prediction_artifact": g["text"], "output_hash": sha256_text(g["text"]), "query_tokens": len(ids), "message_tokens": len(receiver.encode(text)),
146 + "ttft_ms": g["ttft_ms"], "total_wall_ms": (time.perf_counter() - t1) * 1000 + t_chain, "output_usage": {"output_tokens": g["output_tokens"]},
147 + "task_scores": score_synthetic(g["text"], qkeys[q["question_id"]]), "status": "success" if g["text"].strip() else "invalid_output"})
148 + rec["timing_spans"].update({"receiver_prefill_ms": g["prefill_ms"], "ttft_ms": g["ttft_ms"], "decode_ms": g["decode_ms"]})
149 + except Exception as e:
150 + rec["error_code"] = f"{type(e).__name__}: {str(e)[:200]}"
151 + write_record(rec, out_path)
152 + L = hop_meta[h]["ledger"]
153 + log(f"{cid:>28s} {it['document_id']} hop={h} retention={L['retention_rate']:.2f} unsupported={L['unsupported_numbers']}+{L['unsupported_identifiers']} bytes={L['message_bytes']} fetched={fetched_total}")
154 + # store hop messages for audit
155 + Path(ROOT / "runs/raw/chains").mkdir(parents=True, exist_ok=True)
156 + (ROOT / "runs/raw/chains" / f"{cfg['experiment_id']}_{it['document_id']}_{cid.replace('|','_').replace('-','_')}.json").write_text(json.dumps({"hops": hop_msgs, "meta": hop_meta}, ensure_ascii=False, indent=1, default=str))
157 + mx.clear_cache()
158 + log("chains end")
159 + return 0
added src/wp30/cli_extra.py +14 −0
@@ -0,0 +1,14 @@
1 +import sys
2 +def main():
3 + cmd = sys.argv[1]; cfg = sys.argv[2]; resume = "--resume" in sys.argv
4 + if cmd == "chains":
5 + from .chains import run_chains; return run_chains(cfg, resume)
6 + if cmd == "fanin":
7 + from .fanin import run_fanin; return run_fanin(cfg, resume)
8 + if cmd == "router":
9 + from .router import main as f; return f(cfg)
10 + if cmd == "hosted":
11 + from .hosted_run import run as f; return f(cfg, resume)
12 + return 2
13 +if __name__ == "__main__":
14 + sys.exit(main())
modified src/wp30/data/exec_tasks.py +6 −9
@@ -31,9 +31,8 @@ def spec_variants(rng: random.Random, rows):
31 31 spec = (f"Bug report BR-{rng.randint(1000,9999)}: the function net_revenue(rows) must compute the total net revenue of the "
32 32 f"rows list (each row is a dict with keys {COLS}). Rules agreed with the client on {rng.randint(1,28)} {rng.choice(['March','June','September'])} 2031: "
33 33 f"(1) exclude returned rows; (2) exclude rows whose region is '{excl_region}'; (3) exclude rows with fewer than {min_units} units; "
34 − f"(4) rows shipped in more than {thr_days} days receive an additional 5 percentage points of discount on top of discount_pct, capped at 50 %; "
35 − f"(5) line revenue = units * unit_price * (1 - total_discount/100); (6) apply a tax of {tax} % on the sum; "
36 − f"(7) return the result rounded to {rounding} decimals (Python round). The current implementation forgets rule (4) and rule (2).")
34 + f"(4) line revenue = units * unit_price * (1 - discount_pct/100); (5) apply a tax of {tax} % on the sum; "
35 + f"(6) return the result rounded to {rounding} decimals (Python round). The current implementation forgets rule (2) and applies no tax.")
37 36 ref = textwrap.dedent(f"""
38 37 def net_revenue(rows):
39 38 total = 0.0
@@ -44,9 +43,7 @@ def spec_variants(rng: random.Random, rows):
44 43 continue
45 44 if r['units'] < {min_units}:
46 45 continue
47 − d = r['discount_pct'] + (5 if r['shipped_days'] > {thr_days} else 0)
48 − d = min(d, 50)
49 − total += r['units'] * r['unit_price'] * (1 - d / 100.0)
46 + total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)
50 47 total = total * (1 + {tax} / 100.0)
51 48 return round(total, {rounding})
52 49 """)
@@ -77,9 +74,9 @@ def generate_exec_task(seed: int, target_tokens: int = 2000) -> dict:
77 74 doc = ("ENGINEERING TICKET\n\n" + spec + "\n\nSample rows (first 6 of the extract):\n" + sample_rows + "\n\n" +
78 75 "\n".join(filler[: n_fill // 2]) + "\n\nThe existing (buggy) implementation is:\n\n" +
79 76 "def net_revenue(rows):\n total = 0.0\n for r in rows:\n if r['returned']:\n continue\n"
80 − f" if r['units'] < {params['min_units']}:\n continue\n d = r['discount_pct']\n"
81 − " total += r['units'] * r['unit_price'] * (1 - d / 100.0)\n"
82 − f" total = total * (1 + {params['tax']} / 100.0)\n return round(total, {params['rounding']})\n\n" + "\n".join(filler[n_fill // 2:]))
77 + f" if r['units'] < {params['min_units']}:\n continue\n"
78 + " total += r['units'] * r['unit_price'] * (1 - r['discount_pct'] / 100.0)\n"
79 + f" return round(total, {params['rounding']})\n\n" + "\n".join(filler[n_fill // 2:]))
83 80 task = ("Rewrite the Python function net_revenue(rows) so that it implements ALL the agreed rules in the ticket. "
84 81 "Output only Python code defining net_revenue; no explanations, no markdown fences.")
85 82 return {"document_id": f"exec_{seed:05d}_{target_tokens}", "seed": seed, "target_tokens": target_tokens, "text": doc, "char_len": len(doc),
added src/wp30/fanin.py +180 −0
@@ -0,0 +1,180 @@
1 +"""E5: fan-in / composition with conflicting evidence.
2 +
3 +The synthetic document is partitioned into K source agents (project blocks round-robin). Overlap: one project block is
4 +duplicated into a second source with an *earlier* (superseded) revision sentence, so the ledger's precedence rule still
5 +yields the original answers but a naive merge sees a conflict. One source additionally receives an irrelevant block from
6 +another document. The aggregator answers multi-hop/aggregation/revision questions that need several sources.
7 +
8 +Aggregation methods: concat_A4 (concatenated prose summaries), merge_A5 (merged structured records), shared_memory_A6
9 +(reference index per source + metered retrieval by the aggregator), full_context (reference: all source texts concatenated),
10 +and cache_concat_negative_control (KV caches of independently prefilled sources concatenated along the position axis —
11 +positions overlap and no cross-source context exists; reported ONLY as a labelled negative control).
12 +"""
13 +from __future__ import annotations
14 +import json, re, time, yaml, random
15 +from pathlib import Path
16 +import numpy as np
17 +import mlx.core as mx
18 +from .util import ROOT, read_json, sha256_json, sha256_text, utc_now, read_jsonl
19 +from .records import new_record, write_record, trial_key
20 +from .data.prepare import load_doc, load_key
21 +from .data.synthetic import fmt_amt
22 +from .scoring import score_synthetic
23 +from .prompts import SYSTEM_RECEIVER, receiver_user_text
24 +from .backends.mlx_backend import MLXModel
25 +from .cache.sparse import SparseKVCache, cache_tensor_bytes, dense_kv
26 +from .methods import text as TM
27 +
28 +HARDWARE_ID = "laptop-m5max-48g"
29 +
30 +def partition_document(doc_text: str, key: dict, k: int, rng: random.Random, other_doc_text: str) -> list[dict]:
31 + blocks = doc_text.split("\n\n")
32 + header = blocks[0]; review = blocks[1]
33 + proj = [b for b in blocks[2:] if b.startswith("Project ")]
34 + fill = [b for b in blocks[2:] if not b.startswith("Project ")]
35 + sources = [{"source_id": f"S{i+1}", "blocks": [header]} for i in range(k)]
36 + for i, b in enumerate(proj):
37 + sources[i % k]["blocks"].append(b)
38 + for i, b in enumerate(fill):
39 + sources[i % k]["blocks"].append(b)
40 + # overlap with superseded earlier revision: pick a project with >=1 revision
41 + rev_facts = [f for f in key["facts"] if f["attribute"].startswith("revision_1")]
42 + conflict = None
43 + if rev_facts and k >= 2:
44 + f = rev_facts[0]; code = f["entity"]
45 + blk = next((b for b in proj if b.startswith(f"Project {code} ")), None)
46 + if blk:
47 + # earlier date than the first revision: use the proposed date + 1 day (always earlier than revisions)
48 + m = re.search(rf"Project {re.escape(code)} was proposed by .*? on (\d+ \w+ \d{{4}})", blk)
49 + amt = int(re.sub(r"[^\d]", "", f["value"].split()[0])); bogus = fmt_amt(int(amt * 0.93) // 5 * 5)
50 + cur = f["value"].split()[-1]
51 + earlier = f"On {m.group(1)} (preliminary) the budget of {code} was revised to {bogus} {cur}; this preliminary entry is superseded by any later-dated entry."
52 + tgt = (proj.index(blk) + 1) % k
53 + sources[tgt]["blocks"].append(blk.split(". ")[0] + ". " + earlier)
54 + conflict = {"code": code, "bogus_amount": bogus, "source": sources[tgt]["source_id"]}
55 + # irrelevant block from another document
56 + other_proj = [b for b in other_doc_text.split("\n\n") if b.startswith("Project ")]
57 + if other_proj:
58 + sources[-1]["blocks"].append(other_proj[0])
59 + for s in sources:
60 + s["text"] = "\n\n".join(s["blocks"])
61 + return sources, conflict
62 +
63 +def cache_concat_negative_control(receiver: MLXModel, source_texts: list[str]):
64 + """Prefill each source independently (positions 0..T_i), then concatenate K/V along positions. Overlapping positions
65 + and no cross-source attention during prefill make this an INVALID composition; measured as a negative control."""
66 + caches = []
67 + for t in source_texts:
68 + ids = receiver.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": "Document:\n" + t + "\n\n"}])
69 + c, _ = receiver.prefill(ids); caches.append(c)
70 + out = []
71 + for layer in range(len(caches[0])):
72 + ks = [dense_kv(c[layer])[0] for c in caches]; vs = [dense_kv(c[layer])[1] for c in caches]
73 + k = mx.concatenate(ks, axis=2); v = mx.concatenate(vs, axis=2)
74 + out.append(SparseKVCache.from_dense(k, v, logical_offset=max(c[layer].offset for c in caches)))
75 + return out
76 +
77 +def run_fanin(config_path: str, resume: bool = False) -> int:
78 + cfg = yaml.safe_load(open(config_path))
79 + mcfg = yaml.safe_load(open(ROOT / "configs/models.yaml"))["models"]
80 + models = {}
81 + def get(a):
82 + if a not in models: models[a] = MLXModel(mcfg[a]["repo"])
83 + return models[a]
84 + man = read_json(ROOT / "data/manifests/splits.json")
85 + items = [it for it in man["items"] if it["family"] == "synthetic" and it["split"] in cfg["splits"] and it.get("target_tokens") in cfg["lengths"]]
86 + if cfg.get("limit_docs"): items = items[: cfg["limit_docs"]]
87 + out_path = ROOT / "runs/raw" / f"{cfg['experiment_id']}.jsonl"; log_path = ROOT / "runs/logs" / f"{cfg['experiment_id']}.log"
88 + done = {r["trial_id"] for r in read_jsonl(out_path) if r["status"] == "success"} if (resume and out_path.exists()) else set()
89 + protocol_hash = cfg.get("protocol_hash", "unfrozen"); config_hash = sha256_json(cfg)
90 + def log(m):
91 + line = f"{utc_now()} {m}"; print(line, flush=True)
92 + with open(log_path, "a") as f: f.write(line + "\n")
93 + QTYPES = set(cfg.get("question_types", ["multi_hop", "aggregation", "revision", "temporal", "unanswerable"]))
94 + agg = get(cfg["aggregator"]); sender = get(cfg["sender"])
95 + for idx, it in enumerate(items):
96 + doc = load_doc(it["document_id"], "synthetic"); key = load_key(it["document_id"], "synthetic")
97 + other = load_doc(items[(idx + 1) % len(items)]["document_id"], "synthetic")
98 + qkeys = {q["question_id"]: q for q in key["questions"]}
99 + qs = [q for q in doc["questions"] if q["qtype"] in QTYPES]
100 + for k in cfg["fan_in"]:
101 + rng = random.Random(it["seed"] * 7 + k)
102 + sources, conflict = partition_document(doc["text"], key, k, rng, other["text"])
103 + for method in cfg["methods"]:
104 + cid = f"{method}|k={k}"
105 + tks = {q["question_id"]: trial_key(cfg["experiment_id"], it["document_id"], q["question_id"], method, {"k": k, "budget": cfg["budget_tokens"]}, cfg["sender"], cfg["aggregator"], "blind", protocol_hash, 0, "fanin") for q in qs}
106 + if all(t in done for t in tks.values()): continue
107 + t0 = time.perf_counter(); info = {"k": k, "conflict": conflict}
108 + try:
109 + cache = None; material = None; per_q = False
110 + if method == "full_context":
111 + material = "\n\n".join(f"[Source {s['source_id']}]\n{s['text']}" for s in sources); label = "Documents from all sources"
112 + elif method == "concat_A4":
113 + msgs = [TM.a4_summary(s["text"], cfg["budget_tokens"], sender, None) for s in sources]
114 + material = "\n\n".join(f"[Source {s['source_id']} summary]\n{m.text}" for s, m in zip(sources, msgs)); label = "Summaries from several sources"
115 + info["sender_tokens"] = sum(m.sender_usage.get("sender_output_tokens", 0) for m in msgs)
116 + elif method == "merge_A5":
117 + msgs = [TM.a5_structured(s["text"], cfg["budget_tokens"], sender, None) for s in sources]
118 + facts = []
119 + for s, m in zip(sources, msgs):
120 + for f in json.loads(m.text).get("facts", []):
121 + facts.append({"source": s["source_id"], **{kk: vv for kk, vv in f.items() if kk != "span"}})
122 + material = json.dumps({"facts": facts, "rule": "later-dated entries supersede earlier ones"}, ensure_ascii=False); label = "Merged structured evidence from several sources (JSON)"
123 + info["n_facts"] = len(facts)
124 + elif method == "shared_memory_A6":
125 + msgs = [TM.a6_references(s["text"], cfg["budget_tokens"], sender, None) for s in sources]
126 + per_q = True; label = "Reference index from several sources plus retrieved passages"
127 + elif method == "cache_concat_negative_control":
128 + cache = cache_concat_negative_control(agg, [s["text"] for s in sources]); label = None
129 + info["cache_bytes"] = cache_tensor_bytes(cache)
130 + else:
131 + raise NotImplementedError(method)
132 + build_ms = (time.perf_counter() - t0) * 1000
133 + except Exception as e:
134 + log(f"BUILD FAIL {cid} {it['document_id']}: {type(e).__name__}: {str(e)[:200]}"); continue
135 + for q in qs:
136 + tk = tks[q["question_id"]]
137 + if tk in done: continue
138 + rec = new_record(experiment_id=cfg["experiment_id"], trial_id=tk, run_group_id=f"{cfg['experiment_id']}:{cid}", protocol_hash=protocol_hash, config_hash=config_hash,
139 + task_family="synthetic", document_id=it["document_id"], document_hash=it["document_hash"], question_id=q["question_id"], split=it["split"], cluster_id=it["cluster_id"],
140 + sender_model_revision=f"{sender.info.repo_id}@{sender.info.revision}", receiver_model_revision=f"{agg.info.repo_id}@{agg.info.revision}", tokenizer_hashes={"receiver": agg.info.tokenizer_hash},
141 + backend_versions={"mlx": mx.__version__, "mlx_lm": agg.info.mlx_lm_version}, hardware_id=HARDWARE_ID, device_state={}, weight_precision="bf16", method_id=method,
142 + implementation_fidelity="custom" if method != "cache_concat_negative_control" else "n/a", compression_config={"k": k, "budget_tokens": cfg["budget_tokens"], **{kk: vv for kk, vv in info.items() if kk != "conflict"}},
143 + question_access="blind", topology=f"fanin_k{k}", handoff_index=1, reuse_count=len(qs), cache_hit=False, cache_compatibility_hash=None, random_seed=0,
144 + source_tokens_sender=None, source_tokens_receiver=len(agg.encode(doc["text"])), fetched_bytes=0, side_information_bytes=0, timing_spans={"build_ms": build_ms},
145 + input_usage={}, output_usage={}, provider_cached_usage=None, process_memory=None, backend_memory=None, peak_prefill_memory=None, peak_decode_memory=None, mapper_memory=None,
146 + cost_with_source=None, evidence_artifact={"conflict": conflict, "question_type": q["qtype"]}, reference_key_id=f"keys/synthetic/{it['document_id']}.json", status="implementation_error",
147 + error_code=None, log_artifact=str(log_path.relative_to(ROOT)), payload_hash=None, output_hash=None, message_utf8_bytes=None, serialized_payload_bytes=None, cache_tensor_bytes=info.get("cache_bytes"), query_tokens=None, message_tokens=None)
148 + try:
149 + t1 = time.perf_counter()
150 + if cache is not None:
151 + from .cache.sparse import copy_cache
152 + # query tokens = the fresh tail of a full prompt built on the last source (identical tail for every source)
153 + full_ids = agg.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text("Document", sources[-1]["text"], q["question"], "synthetic")}])
154 + pre_ids = agg.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": "Document:\n" + sources[-1]["text"] + "\n\n"}])
155 + n = 0
156 + while n < min(len(pre_ids), len(full_ids)) and pre_ids[n] == full_ids[n]: n += 1
157 + q_ids = full_ids[n:]
158 + g = agg.generate_from_cache(copy_cache(cache), q_ids, max_tokens=48)
159 + rec.update({"serialized_payload_bytes": info["cache_bytes"], "message_utf8_bytes": sum(len(s["text"].encode()) for s in sources), "query_tokens": len(q_ids)})
160 + else:
161 + mat = material
162 + if per_q:
163 + pieces = []; fb = 0; lk = 0
164 + for m in msgs:
165 + mt, ri = TM.a6_retrieve(m, q["question"], cfg.get("max_fetch_bytes", 1500), cfg.get("max_lookups", 3)); pieces.append(mt); fb += ri["fetched_bytes"]; lk += ri["lookups"]
166 + mat = "\n\n".join(pieces); rec["fetched_bytes"] = fb; rec["input_usage"] = {"lookups": lk}
167 + ids = agg.chat_prompt_tokens([{"role": "system", "content": SYSTEM_RECEIVER}, {"role": "user", "content": receiver_user_text(label, mat, q["question"], "synthetic")}])
168 + g = agg.generate_text(ids, max_tokens=48)
169 + rec.update({"serialized_payload_bytes": len(mat.encode("utf-8")), "message_utf8_bytes": len(mat.encode("utf-8")), "query_tokens": len(ids), "message_tokens": len(agg.encode(mat)), "payload_hash": sha256_text(mat)})
170 + rec.update({"prediction_artifact": g["text"], "output_hash": sha256_text(g["text"]), "ttft_ms": g["ttft_ms"], "total_wall_ms": (time.perf_counter() - t1) * 1000 + build_ms,
171 + "output_usage": {"output_tokens": g["output_tokens"]}, "task_scores": score_synthetic(g["text"], qkeys[q["question_id"]]), "status": "success" if g["text"].strip() else "invalid_output"})
172 + rec["timing_spans"].update({"ttft_ms": g["ttft_ms"], "decode_ms": g["decode_ms"], "receiver_prefill_ms": g.get("prefill_ms", 0.0)})
173 + except Exception as e:
174 + rec["error_code"] = f"{type(e).__name__}: {str(e)[:200]}"
175 + write_record(rec, out_path)
176 + sc = rec["task_scores"] or {}
177 + log(f"{cid:>36s} {it['document_id']} {q['question_id']:>18s} {rec['status']:>8s} corr={sc.get('correct')} bytes={rec['serialized_payload_bytes']} ans={json.dumps((rec['prediction_artifact'] or '')[:50])}")
178 + mx.clear_cache()
179 + log("fanin end")
180 + return 0
modified src/wp30/runner.py +14 −6
@@ -90,18 +90,24 @@ class DocContext:
90 90 self._dense = None; self.build_ms = None; self.prefix_ids = None; self.full_ids_fn = None
91 91 self.sys = SYSTEM_RECEIVER
92 92 def prefix_tokens(self):
93 + """Receiver prefix = chat tokens up to (and including) the document and the blank line before 'Question:'.
94 + Computed as the longest common token prefix between the prefix text and a full prompt with a canonical question."""
93 95 if self.prefix_ids is None:
94 96 pre_txt = self.receiver.tokenizer.apply_chat_template([{"role": "system", "content": self.sys}, {"role": "user", "content": "Document:\n" + self.doc_text + "\n\n"}], add_generation_prompt=False, tokenize=False)
95 − self.prefix_ids = self.receiver.encode(pre_txt)
97 + pre = self.receiver.encode(pre_txt)
98 + full = self.receiver.chat_prompt_tokens([{"role": "system", "content": self.sys}, {"role": "user", "content": receiver_user_text("Document", self.doc_text, "What is stated?", self.family)}])
99 + n = 0
100 + while n < min(len(pre), len(full)) and pre[n] == full[n]:
101 + n += 1
102 + self.prefix_ids = full[:n]
96 103 return self.prefix_ids
97 104 def split_ids(self, question: str):
98 105 """Full chat tokens for (doc, question); returns (prefix_ids, query_ids) with prefix a true token prefix."""
99 106 full = self.receiver.chat_prompt_tokens([{"role": "system", "content": self.sys}, {"role": "user", "content": receiver_user_text("Document", self.doc_text, question, self.family)}])
100 107 pre = self.prefix_tokens()
101 − n = 0
102 − while n < min(len(pre), len(full)) and pre[n] == full[n]:
103 − n += 1
104 − return full[:n], full[n:]
108 + if full[:len(pre)] != pre:
109 + raise RuntimeError("prefix_boundary: question tokenization altered the document prefix")
110 + return pre, full[len(pre):]
105 111 def dense_cache(self, prefix_ids):
106 112 if self._dense is None or self._dense_len != len(prefix_ids):
107 113 reset_peak()
@@ -130,7 +136,8 @@ def run(config_path: str, resume: bool = False, limit: int | None = None, dry_ru
130 136 done.add(r["trial_id"])
131 137 items = _select_items(cfg)
132 138 get_model, mcfg = _load_models(cfg)
133 − gen_max = cfg.get("generation", {}).get("max_tokens", 48)
139 + gen_cfg = cfg.get("generation", {})
140 + gen_max = gen_cfg.get("max_tokens", 48)
134 141 n_done = n_run = n_fail = 0
135 142 log_path = ROOT / "runs/logs" / f"{cfg['experiment_id']}.log"
136 143 log_path.parent.mkdir(parents=True, exist_ok=True)
@@ -145,6 +152,7 @@ def run(config_path: str, resume: bool = False, limit: int | None = None, dry_ru
145 152 for it in items:
146 153 fam = it["family"]; doc = load_doc(it["document_id"], fam); key = load_key(it["document_id"], fam)
147 154 qs = doc["questions"][: cfg.get("max_questions_per_doc") or None]
155 + gen_max = gen_cfg.get("per_family", {}).get(fam, gen_cfg.get("max_tokens", 48))
148 156 qkeys = {q["question_id"]: q for q in key["questions"]}
149 157 for pair in cfg["model_pairs"]:
150 158 receiver = get_model(pair["receiver"]); sender = get_model(pair["sender"])
151 159
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere