SPB Git

spb/wp30_uqo

Public
1.6 KB · 44 lines yaml
1experiment_id: E1_pilot2splits: [dev]3families: [synthetic, exec, public_qasper, public_2wikimqa]4lengths: [2000, 8000]5limit_docs_per_family: 26max_questions_per_doc: 57model_pairs:8  - {sender: qwen3b, receiver: qwen3b}9  - {sender: llama3b, receiver: llama3b}10generation: {max_tokens: 48, per_family: {exec: 320}}11seeds: [0]12conditions:13  - {method: A0}14  - {method: A1}15  - {method: A2, budget_tokens: 128}16  - {method: A2, budget_tokens: 512}17  - {method: A3, budget_tokens: 128, access: blind}18  - {method: A3, budget_tokens: 512, access: blind}19  - {method: A3, budget_tokens: 512, access: aware}20  - {method: A4, budget_tokens: 128, access: blind}21  - {method: A4, budget_tokens: 512, access: blind}22  - {method: A4, budget_tokens: 512, access: aware}23  - {method: A5, budget_tokens: 128, access: blind}24  - {method: A5, budget_tokens: 512, access: blind}25  - {method: A5, budget_tokens: 512, access: aware}26  - {method: A6, budget_tokens: 512, access: blind, max_fetch_bytes: 2000, max_lookups: 6}27  - {method: A7, codec: zstd}28  - {method: A8, budget_tokens: 512, access: blind}29  - {method: B0}30  - {method: B1}31  - {method: B2, bits: 8}32  - {method: B2, bits: 4}33  - {method: B2, bits: 2, group_size: 32}34  - {method: B3, keep_frac: 0.5}35  - {method: B3, keep_frac: 0.25}36  - {method: B3, keep_frac: 0.125}37  - {method: B4, keep_frac: 0.25}38  - {method: B5, keep_frac: 0.5, access: aware}39  - {method: B5, keep_frac: 0.25, access: aware}40  - {method: B5, keep_frac: 0.125, access: aware}41  - {method: B5, keep_frac: 0.25, access: blind}42  - {method: B6, keep_frac: 0.25, access: aware}43  - {method: B9, keep_frac: 0.25, bits: 8, access: aware, policy: attention}44
Files · 2ccc6b7e52385b2438c0f3e62e88226009fb4f9f
Menu

Theme