loading reference mlx-community/Qwen3-8B-bf16 …

Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.

Fetching 12 files:  17%|█▋        | 2/12 [00:00<00:00, 15.49it/s]
Fetching 12 files:  42%|████▏     | 5/12 [00:00<00:00, 18.46it/s]
Fetching 12 files:  58%|█████▊    | 7/12 [00:00<00:00, 17.87it/s]
Fetching 12 files:  67%|██████▋   | 8/12 [00:20<00:00, 17.87it/s]
Fetching 12 files:  75%|███████▌  | 9/12 [00:38<00:20,  6.86s/it]
Fetching 12 files:  83%|████████▎ | 10/12 [01:43<00:37, 18.88s/it]
Fetching 12 files:  92%|█████████▏| 11/12 [01:44<00:14, 14.68s/it]
Fetching 12 files: 100%|██████████| 12/12 [01:50<00:00,  9.21s/it]
  generated code
  generated math
  generated chat
  generated french
  generated english
  generated reasoning
reference generation done in 194s (48 trajectories)
quantizing to 3-bit (group 64) …

Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]
Fetching 12 files: 100%|██████████| 12/12 [00:00<00:00, 7472.04it/s]
  3-bit: agree=0.8527 AUROC=0.883 esc@99%=0.4098307291666667
quantizing to 4-bit (group 64) …

Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]
Fetching 12 files: 100%|██████████| 12/12 [00:00<00:00, 2337.96it/s]
  4-bit: agree=0.9212 AUROC=0.920 esc@99%=0.22607421875
quantizing to 8-bit (group 64) …

Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]
Fetching 12 files: 100%|██████████| 12/12 [00:00<00:00, 6075.77it/s]
  8-bit: agree=0.9867 AUROC=0.981 esc@99%=0.015950520833333332

wrote /Users/simon-pierreboucher/Desktop/localvm-research/results/expG_decision_stability/20260812T051129Z/results.json